NAVAL 

POSTGRADUATE 

SCHOOL 


MONTEREY,  CALIFORNIA 

THESIS 


EFFECTIVE  TEAMING  OF  AIRBORNE  AND  GROUND 
ASSETS  FOR  SURVEILLANCE  AND  INTERDICTION 


Thesis  Co- Advisors: 
Second  Reader: 


by 

Mark  J.  Muratore 
June  2010 


Timothy  H.  Chung 
Rachel  T.  Johnson 
Chad  W.  Seagren 


Approved  for  public  release;  distribution  is  unlimited 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to 
comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


11 


Approved  for  public  release;  distribution  is  unlimited 


EFFECTIVE  TEAMING  OF  AIRBORNE  AND  GROUND  ASSETS  FOR 
SURVEILLANCE  AND  INTERDICTION 


Mark  J.  Muratore 

Major,  United  States  Marine  Corps 
B.A.  Syracuse  University,  1992 

Submitted  in  partial  fulfillment  of  the 
requirements  for  the  degree  of 

MASTER  OF  SCIENCE  IN  OPERATIONS  RESEARCH 

from  the 

NAVAL  POSTGRADUATE  SCHOOL 
June  2010 


Author: 


Mark  J.  Muratore 


Approved  by:  Timothy  H.  Chung 

Thesis  Co-Advisor 


Rachel  T.  Johnson 
Thesis  Co-Advisor 


Chad  W.  Seagren 
Second  Reader 


Robert  F.  Dell 

Chair,  Department  of  Operations  Research 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


IV 


ABSTRACT 


As  Unmanned  Aerial  Vehicles  (UAVs)  become  more  prevalent  on  the  battlefield,  ground  forces 
will  have  to  increasingly  rely  on  them  for  intelligence,  surveillance,  and  reconnaissance  (ISR), 
as  well  as  target  marking,  and  overwatch  operations.  The  Situational  Awareness  for  Surveil¬ 
lance  and  Interdiction  Operations  (S  ASIO)  simulation  analysis  tool  uses  Design  of  Experiments 
(DOX)  to  study  of  aspects  of  UAV  surveillance  characteristics  in  conjunction  with  ground-based 
interdiction  teams.  The  goal  is  to  reduce  the  time  required  to  intercept  and  capture  targets  of 
interest.  Through  screening  analysis,  significant  factors  can  be  determined  to  build  a  model  that 
will  provide  a  ground  commander  with  insights  to  aid  in  the  tactical  employment  of  his  assets. 
We  will  examine  different  teaming  strategies  and  coordination  measures  between  searching  and 
interdicting  assets  in  order  to  study  the  effectiveness  of  the  interdictor  possessing  an  organic, 
tracker  UAV.  The  objective  of  this  research  is  to  quantify  the  benefit  or  penalty  of  an  additional 
UAV  asset  that  is  organic  to  a  quick  reaction  force,  in  the  context  of  the  overall  surveillance  and 
interdiction  operation. 
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Executive  Summary 


As  Unmanned  Aerial  Vehicles  (UAVs)  become  more  prevalent  on  the  battlefield,  ground  forces 
will  have  to  increasingly  rely  on  them  for  intelligence,  surveillance,  and  reconnaissance  (ISR), 
as  well  as  target  marking,  and  overwatch  operations.  The  Situational  Awareness  for  Surveil¬ 
lance  and  Interdiction  Operations  (S  ASIO)  simulation  analysis  tool  uses  Design  of  Experiments 
(DOX)  to  allow  for  the  study  of  aspects  of  UAVs  surveillance  characteristics  in  conjunction  with 
ground-based  interdiction  teams  to  aid  in  reducing  the  time  required  to  intercept  and  capture 
targets  of  interest.  Through  screening  analysis,  significant  factors  can  be  determined  to  build 
a  model  that  will  provide  a  ground  commander  with  insights  to  aid  in  the  tactical  employment 
of  his  assets.  We  will  examine  different  teaming  strategies  and  coordination  measures  between 
searching  and  interdicting  assets  in  order  to  study  the  effectiveness  of  the  interdictor  possessing 
an  organic,  tracker  UAV.  The  objective  of  this  research  is  to  quantify  the  benefit  or  penalty  of 
an  additional  UAV  asset  that  is  organic  to  a  quick  reaction  force,  in  the  context  of  the  overall 
surveillance  and  interdiction  operation. 

Utilizing  design  of  experiments  and  statistical  methods,  we  focused  on  identifying  the  signifi¬ 
cant  factors  that  impact  the  Percentage  of  Targets  Cleared  and  the  Percentage  of  Time  Surveyor 
UAV  Performs  Search  for  three  different  team  types.  These  team  types  were  (1)  Surveyor,  (2) 
Surveyor/Tracking,  and  (3)  Surveyor  with  Tracker.  We  were  able  to  determine,  in  all  cases,  that 
the  factors  with  the  greatest  effect  were  the  Search  Area  size  and  Team  Type.  Holding  all  other 
factors  constant,  we  determined  that  as  the  Search  Area  increases,  the  Percentage  of  Targets 
Cleared  decreases  and  the  Percentage  of  Time  Surveyor  UAV  Performs  Search  increases.  We 
also  discovered  that  possessing  a  tracking  capability  is  clearly  superior  to  no  tracking  capability. 
Of  particular  interest  is  that  there  was  no  significant  difference  between  the  Surveyor/Tracking 
and  Surveyor  with  Tracker  Team  Types.  Therefore,  we  conclude  that  possessing  an  augmented 
tracking  capability  does  not  increase  the  Percentage  of  Targets  Cleared. 

We  also  discovered  that  the  Interdictor  Transit  Time  was  a  significant  factor  for  our  model.  The 
time  required  to  reach  targets  of  interest  was  the  limiting  factor  in  clearing  those  targets.  We 
recommend  studying  various  forms  of  mobility  with  differing  speeds  to  determine  which  would 
increase  the  Percentage  of  Targets  Cleared.  Through  field  experimentation  at  Camp  Roberts 
CA,  we  identified  various  real-world  constraints  that  effect  the  response  variables.  Specifically, 
weather  considerations  and  QRF  proficiency  in  UAV  operations  will  impact  the  responses.  Nei- 


ther  of  these  considerations  were  modeled  for  our  simulation  and  are  worthy  of  future  study. 


The  SASIO  model  can  be  used  for  any  future  studies  involving  teaming.  It  is  recommended  for 
future  Naval  studies  in  the  use  of  Unmanned  Underwater  Vehicles  (UUVs)  and  Marine  Corps 
studies  in  the  use  of  Cargo  UAS  as  well  as  Unmanned  Ground  Vehicles  (UGVs). 
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CHAPTER  1 : 
Introduction 


1.1  Background 

The  Marine  Corps  Vision  and  Strategy  2025  [1]  clearly  details  the  Commandant’s  vision  of  the 
future  Corps  and  describes  his  plan  for  creating  the  Marine  Corps  of  2025.  Two  important  facets 
of  this  vision  state  the  Marine  Corps  will  deploy  forward  with  relevant  and  timely  capabilities 
and  will  be  lean,  agile,  and  adaptable.  As  part  of  this  vision,  the  Deputy  Commandant  for  Avi¬ 
ation  has  outlined  goals  to  ensure  the  Aviation  Combat  Element  (ACE)  of  the  Marine  Corps  is 
aligned  with  Strategy  2025.  Two  of  these  goals  are  :  (1)  execute  the  planned  Type/Model/Series 
transition  strategies  from  the  legacy  platforms  to  the  next  generation  platforms,  and  (2)  improve 
warfighting  integration  by  developing  new  concepts  of  operations  to  leverage  our  transforma¬ 
tional  systems. 

To  support  these  goals,  the  Marine  Corps  has  pursued  developments  with  unmanned  aerial  ve¬ 
hicles  to  increase  the  force-multiplying  capabilities  that  these  enhanced,  multispectral  (Electro- 
Optical/Infrared  and  Synthetic  Aperture  Radar)  systems  bring  to  the  fight.  Newly  emergent 
concepts  for  unmanned  aerial  systems  (UASs)  employment  will  continue  to  enhance  and  ex¬ 
tend  the  lethal  and  nonlethal  capabilities  of  the  Marine  Air  Ground  Task  Force  (MAGTF).  Joint 
Force  Commanders  will  attain  new  levels  of  battlespace  command  and  control  and  situational 
awareness  by  applying  these  newly  emergent  concepts. 

A  Concept  of  Operations  (CONOPS)  for  the  United  States  Marine  Corps  Unmanned  Aircraft 
Systems,  Family  of  Systems  (FoS)  [2]  was  developed  to  provide  fundamental  guidance  for 
initial  concepts,  tactics,  techniques,  and  procedures  (TTPs)  in  order  to  meet  the  need  of  persis¬ 
tent  Intelligence,  Surveillance,  and  Reconaissance  (ISR)  on  today’s  battlefield.  As  part  of  this 
CONOPS,  three  groups  of  unmanned  aerial  vehicles  (UAVs)  have  been  identified  for  employ¬ 
ment  by  the  USMC.  They  are  Group  1,  Group  3,  and  Group  4  systems. 

Previous  studies  examined  the  interactions  of  UAVs  amongst  themselves  in  the  context  of  target 
acquisition  and  engagement.  The  Navy  is  studying  future  applications  and  use  of  UAVs.  This  is 
highlighted  by  the  Chief  of  Naval  Operations  (CNO)  most  recent  Strategic  Studies  Group  (SSG) 
XXVIII  study  on  autonomous  systems,  entitled  “The  Unmanned  Imperative.”  Specifically,  the 
SSG  was  tasked  to  show  how  unmanned  and  manned  systems  interact  and  to  optimize  the  com- 
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mand  and  control  structure  to  better  integrate  unmanned  naval  assets  with  manned  systems.  The 
SSG  believes  this  integration  will  allow  the  Naval  services  to  overcome  the  challenges  of  the 
21st  century.  In  order  to  accomplish  this,  UAVs  need  to  become  more  capable.  Future  use  of 
UAVs  will  require  them  to  be  distributed  globally,  dispersed  geographically,  and  have  disag¬ 
gregated  functionality.  Global  distribution  allows  the  commander  to  gain  regional  knowledge 
through  persistent  ISR.  Geographical  dispersion  provides  for  greater  battlespace  awareness. 
Disaggregated  functionality  refers  to  tailored  functions  and  capabilities  based  on  mission  need. 

One  of  the  critical  capabilities  of  the  UAV  is  autonomy.  Autonomous  systems  can  be  thought 
of  as  possessing  different  “levels  of  autonomy”  with  some  degree  of  human  interaction  com¬ 
bined  with  machine  automation.  Future  CONOPS  for  all  services  will  increasingly  rely  on  this 
capability  in  some  form,  as  many  current  UAVs,  such  as  the  Global  Hawk,  already  operate  with 
some  degree  of  autonomy.  A  high  level  of  automation  will  be  required  for  future  concepts  that 
include  large  numbers  of  UAVs.  In  many  large-scale  operations  where  high  numbers  of  UAVs 
may  operate,  the  ability  to  self-coordinate  may  be  needed  due  to  the  increased  difficulty  for 
operators  to  control  multiple  teams  of  UAVs. 

However,  there  is  a  pitfall  if  the  number  of  UAVs  and  their  uses  are  increased  without  viewing 
them  as  part  of  an  overall  distributed  force.  Many  unseen  and  unexploited  opportunities  for 
the  use  of  these  systems  may  be  missed.  Thus,  it  is  important  that  these  systems  be  utilized 
in  such  a  way  as  to  maximize  the  integration  of  unmanned  and  manned  systems.  Studying  the 
integration  and  future  employment  options  among  UAVs  and  the  interdiction  teams  they  are 
tasked  to  support  is  just  a  first  step  in  this  process.  If  ground  interdiction  teams  are  augmented 
with  an  organic  tracking  capability  ( the  ability  to  track  targets  on  their  own)  in  the  form  of  a 
man-portable  UAV,  the  potential  exists  to  more  rapidly  acquire  and  track  a  target,  which  would 
allow  the  surveyor  UAV  to  be  released  from  tracking  and  return  to  its  primary  mission  of  ISR 
and  target  acquisition.  It  is  believed  the  natural  result  of  this  interaction  would  be  an  increase 
in  the  number  of  targets  acquired  and  captured  during  interdiction  missions. 

1.1.1  Overview  of  USMC  UAV  Family  of  Systems  (FOS) 

The  essence  of  the  USMC  FOS  is  to  provide  the  MAGTF  with  dedicated  operational  capabilities 
focused  on  battlespace  awareness  and  force  application  while  enabling  enhanced  command  and 
control  (C2)  throughout  the  range  of  diverse  military  operations.  In  order  to  accomplish  this, 
the  Marine  Corps  will  need  to  acquire  significant  numbers  of  smaller  UAVs  that  will  be  organic 
to  the  ground  combat  elements  table  of  equipment.  These  smaller  UAVs  will  become  part  of 
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UAS  Category 

Max  Gross 

Takeoff  Weight 

Normal  Operating 
Altitude  (Ft) 

Airspeed 

Representative  UAS 

Group  1 

<  20  pounds 

<  1200  AGL 

<100  Kts 

Wasp,  Raven  B 

Group  2 

21-55  pounds 

<  3500  AGL 

<  250  Kts 

Scan  Eaqle 

Group  3 

<  1320  pounds 

<18,000  MSL 

RQ-7  Shadow 

Group  4 

>  1320  pounds 

Any 

Airspeed 

RQ-9  Reaper 

Group  5 

>  18,000  MSL 

RQ-4  Global  Hawk 

Table  1.1:  Joint  UAS  Category 


the  different  “Groups”  of  UASs  within  the  Marine  Corps  based  on  their  capabilities.  The  group 
definitions  have  been  categorized  by  the  Joint  UAS  Center  of  Excellence  based  on  weight, 
operating  altitude,  and  airspeed  (see  Table  1.1).  The  greater  these  attributes,  the  higher  the 
group  number.  Within  the  Marine  Corps,  Group  1  systems  are  micro  and  small  UAVs,  Group  3 
systems  are  small  tactical  UAS  (STUAS),  and  Group  4  systems  are  Marine  Corps  tactical  UASs 
(MCTUAS).  The  Marine  Corps  currently  does  not  have  a  Group  2  UAS  requirement. 

Group  1  systems  include  the  RQ-1  IB  Raven  and  Wasp.  Wasp  is  organic  to  the  infantry  company 
and  employed  at  the  platoon  or  squad  level.  It  is  designed  to  be  a  small,  rugged,  man-portable 
system  for  employment  at  the  tactical  unit  level  for  ISR  missions.  The  Raven  is  organic  to  the 
infantry,  artillery,  armor,  light  reconnaissance,  and  amphibious  assault  battalions  and  employed 
at  the  company  or  battery  level.  It  provides  the  company  commander  with  enhanced  situational 
awareness  and  observation  of  targets  beyond  the  company’s  line  of  sight  (LOS).  It  is  larger 
than  Wasp;  however,  both  systems  utilize  common  control  stations  and  thus  operation  of  these 
systems  is  similar.  These  systems  are  designed  for  close-in  reconnaissance,  surveillance,  and 
target  acquisition  in  support  of  infantry  squads,  platoons,  companies,  and  battalions. 

Group  3  systems  are  designed  to  be  simple,  transportable  and  rugged,  yet  capable  of  producing 
actionable  intelligence  for  the  maneuver  commander.  Group  3  systems  will  provide  dedicated 
UAV  support  at  the  battalion  level.  Each  system  will  have  the  following  capabilities:  electro- 
optical/infrared  (EO/IR)  sensors,  integrated  IR  marker  and  laser  range  finder/designator,  and 
communication  relay  packages.  These  systems  are  designed  to  be  shipboard  compatible  with 
an  unobstructed  LOS  operating  radius  of  approximately  fifty  nautical  miles  (NM),  and  operate 
up  to  altitudes  of  15,000  feet  for  extended  periods  of  up  to  ten  hours.  Group  3  UAYs  will  be 
assigned  to  the  Marine  Unmanned  Aerial  Vehicle  Squadrons  (VMU)  within  the  Marine  Aircraft 
Wing  (MAW). 

Group  4  missions  are  currently  performed  by  a  Group  3  system  according  to  the  definition. 
The  future  Group  4  system  will  be  the  MAGTF’s  long-range,  high-speed  UAV  with  mission 
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sets  that  include  C2,  ISR,  communication  relay,  target  acquisition  and  marking,  and  precision 
strike.  This  system  would  be  employed  at  the  Marine  Expeditionary  Brigade  (MEB)  or  Marine 
Expeditionary  Force  (MEF)  levels.  This  platform  may  have  future  missions  of  performing  the 
role  of  a  cargo  UAV  or  as  an  electronic  warfare  (EW)  platform. 

1.1.2  Future  Vision  of  the  USMC  UAV  FOS 

The  UAS  FoS  provides  the  Marine  Air-Ground  Task  Force  (MAGTF)  and  its  subor¬ 
dinate  units  dedicated  operational  and  tactical,  interoperable,  integrated  and  tailored 
Battlespace  Awareness  and  Force  Application  capabilities  while  enabling  enhanced 
Command,  Control,  and  Communications  throughout  the  range  of  military  opera¬ 
tions.  USMC  UAV  perform  key  roles  in  all  Six  Functions  of  Marine  Aviation: 

Aerial  Reconnaissance,  Electronic  Warfare  (EW),  Offensive  Air  Support  (OAS), 
Assault  Support,  Anti-air  Warfare  (AAW),  and  Control  of  Aircraft  and  Missiles. 

[2] 


The  above  statement  is  a  sign  of  the  future  employment  of  UAVs  within  the  Marine  Corps. 
UAVs  will  be  expected  to  perform  and  succeed  in  the  six  functions  of  Marine  Aviation.  Cur¬ 
rently,  with  the  exception  of  aerial  reconnaissance,  the  six  functions  are  performed  by  manned 
fixed  and  rotary  wing  aircraft.  The  vision  is  designed  to  not  only  save  lives,  but  to  provide 
for  a  high  degree  of  interoperability  among  these  systems  and  ground  units  to  accomplish  the 
mission.  Some  of  these  future  concepts  include  Cargo  UAS,  Casualty  Evacuation  (CASE- 
VAC)  UAS,  Manned-Unmanned  Teaming  (MUT),  Unmanned  Combat  Air  System  (UCAS), 
cross  cueing  (adding  information  about  a  target  of  interest),  and  intelligence  fusion  (combing 
all  information  about  a  target). 

Cargo  UAS  was  envisioned  as  a  need  to  reduce  the  number  of  convoys  that  are  exposed  to 
the  improvised  explosive  device  (IED)  threat.  By  utilizing  UASs  for  logistic  support,  potential 
IED  attacks  can  be  avoided  and  resupply  can  happen  much  more  quickly,  due  to  direct  routing 
of  the  UAV  and  its  speed.  In  a  similar  manner,  UASs  can  be  utilized  to  evacuate  casualties 
from  the  combat  zone  and  deliver  them  to  field  hospitals  throughout  the  AO.  MUT  is  the  focus 
of  this  thesis  and  will  be  discussed  in  detail  throughout.  UCAS  is  envisioned  to  reduce  the 
need  to  send  manned  aircraft  into  hostile  areas.  These  UAVs  will  be  able  to  deliver  the  same 
precision  weapons  as  manned  aircraft.  They  will  also  be  smaller,  lighter,  and  stealthier  than 
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manned  aircraft,  thus  reducing  their  radar  cross-section  and  vulnerability  to  detection  by  enemy 
radar.  This  low  observability  translates  into  fewer  fatalities  in  the  combat  zone.  Cross  cueing 
and  intelligence  fusion  are  concepts  that  are  being  practiced  today  but  with  the  development 
of  new  technologies,  these  capabilities  will  become  even  more  powerful.  A  real-time,  fused 
intelligence  picture  will  allow  the  analyst  and  warfighter  to  distinguish  targets  in  cluttered,  urban 
environments  more  easily,  thereby  increasing  the  likelihood  of  overall  mission  success. 

With  many  of  these  concepts  in  mind,  we  will  examine  a  small  part  of  the  future  requirements 
in  this  thesis.  We  will  seek  to  examine  MUT,  specifically  Group  1  and  Group  3  interactions 
with  a  quick  reaction  force  (QRF)  or  interdiction  team,  to  aid  in  the  development  of  a  CONOPS 
for  UAS  employment  as  described  in  the  next  section. 


1.2  Problem  Statement 

The  Situational  Awareness  for  Surveillance  and  Interdiction  Operations  (SASIO)  software  pro¬ 
vides  an  analysis  tool  and  is  designed  to  study  mission  characteristics  and  performance  involv¬ 
ing  surveillance  assets  such  as  UAVs  in  conjunction  with  interdiction  assets  such  as  ground- 
based  QRF  teams.  We  will  examine  different  teaming  strategies  and  coordination  measures 
between  searching  and  interdicting  assets  in  order  to  study  the  effectiveness  of  the  interdictor 
possessing  an  organic,  tracker  UAV.  The  team’s  effectiveness  is  measured  by  the  number  of 
successful  interdictions,  i.e.,  detection,  interception,  and  capture,  of  targets  of  interest. 

Currently,  there  is  a  one-way  flow  of  information,  in  that  a  ground  unit  may  request  an  airborne 
asset  to  assist  with  its  mission,  but  they  do  not  have  direct  control  over  that  asset.  It  is  either 
piloted  remotely  from  the  Tactical  Operations  Center  (TOC)  or  by  aircrew  of  a  fixed  or  rotary 
wing  aircraft.  As  UAVs  become  more  prevalent  on  the  battlefield,  ground  forces  will  have  to 
increasingly  rely  on  them  for  ISR  as  well  as  target  marking,  overwatch  operations,  and  perhaps 
even  enemy  engagement.  There  exists  a  need  within  all  services  for  new  or  better  TTPs  and 
CONOPS  for  effectively  teaming  these  ground  units  with  organic  UAVs. 

The  objective  of  this  research  is  to  quantify  the  benefit  or  penalty  of  an  additional  UAV  asset  in 
the  context  of  the  overall  surveillance  and  interdiction  operation  using  the  SASIO  model. 


1.3  Research  Questions 

The  objectives  of  this  thesis  research  are  guided  by  the  following  research  questions: 
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•  Is  it  better  for  an  interdiction  team  to  possess  an  organic  “augmented  tracking”  capability 
in  the  context  of  time  to  intercept  and  number  of  targets  captured? 

•  What  are  the  significant  factors  that  produce  teaming  strategies  that  result  in  the  greatest 
number  of  hostile  targets  captured? 


1.4  Literature  Review 

As  UAVs  become  integrated  on  the  battlefield,  it  is  increasingly  important  to  operate  them  in 
a  highly  efficient  manner.  Many  concepts  apply  to  future  UAV  operations.  One  concept  is  the 
efficient  routing  of  UAVs  in  the  Area  of  Operations  (AO).  Efficient  routing  needs  to  take  into 
account  terrain,  mission,  search  pattern,  number  of  targets,  and  the  network  structure  of  the 
AO.  This  concept  has  been  studied  in  [3],  where  the  authors  developed  a  decision  support  tool 
utilizing  a  dynamic  program  that  provides  efficient  search  routes  for  multiple  UAVs  searching 
for  multiple  targets  on  a  known  graph  of  nodes  and  arcs.  This  problem  was  extended  to  the 
study  of  path  planning  for  multiple  UAVs  [4].  In  this  study,  the  authors  addressed  the  problem  of 
generating  feasible  routes  for  multiple  UAVs  in  a  target  rich  environment.  An  anytime  algorithm 
was  developed  using  particle  swarm  optimization  that  results  in  routes  whose  quality  increases 
with  an  increase  in  the  computation  time. 

A  second  concept  of  UAV  employment  is  that  of  sensing  or  target  acquisition.  Two  significant 
factors  that  affect  sensing  are  the  probability  that  a  target  exists  at  a  given  location  and  the 
probability  of  the  UAV  seeing  and  identifying  a  target.  One  can  use  the  number  of  hostile 
targets  identified  as  a  measure  of  effectiveness  (MOE)  for  a  given  employment  strategy.  [5] 
investigated  the  value  of  predictive  target  assignment  as  a  function  of  the  number  of  unknown 
targets  and  number  of  UAVs.  Target  neutralization  time  (i.e.,  the  time  needed  to  neutralize 
targets)  and  total  mission  time  (i.e.,  the  time  needed  to  destroy  all  targets)  were  the  MOEs  used. 
It  was  found  that  utilizing  a  prediction  algorithm  for  target  locations  helped  the  cooperative 
UAV  teams  locate  targets.  As  targets  were  located,  they  were  stored  in  a  database  used  by 
the  UAV  team  to  update  target  knowledge  and  thus  update  the  prediction  algorithm.  As  the 
target  database  increased,  the  predictions  improved.  In  this  study,  the  nominal  density  of  targets 
within  the  AO  will  be  known  a  priori  and  we  will  utilize  different  search  algorithms  as  factors 
for  our  MOE.  [6]  studied  the  value  of  utilizing  probabilistic  information  about  reports  of  object 
detections  and  incorporated  this  information  into  a  database  that  includes  probabilities  of  an 
object’s  existence  as  well  as  probabilities  of  its  location.  This  aids  in  the  discrimination  of  false 
and  real  objects. 
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A  natural  extension  of  UAV  teaming  is  the  concept  of  MUT.  MUT  is  a  concept  that  takes 
advantage  of  a  UAV’s  ability  to  detect  targets  and  a  ground  team’s  ability  to  interdict  those 
targets.  It  is  precisely  this  concept  that  we  study  in  this  thesis.  An  integer  linear  program 
was  developed  to  optimize  the  employment  and  deployment  of  UAVs  integrated  with  Special 
Forces  [7].  The  goal  of  this  model  was  to  assist  commanders  in  determining  suitable  locations 
for  mobile  control  centers  (MCC)  and  ground  control  units  (GCU)  as  well  as  optimizing  the 
search  areas  for  the  UAVs.  The  model  resulted  in  50%  more  target  detections  than  manual 
plans  generated  by  experienced  commanders. 

In  order  for  success  on  the  battlefield,  UASs  cannot  be  limited  to  operate  in  isolation.  There 
must  exist  a  level  of  coordination  and  integration  with  ground  units.  This  problem  was  studied 
in  an  attempt  to  quantify  the  benefit  of  small,  hand-launched  UAVs  as  border  patrol  agents 
attempted  to  classify  and  capture  illegal  aliens  at  border  crossing  sites  [8].  The  MOEs  used 
for  this  study  were  the  number  of  illegal  immigrants  classified  and  captured  and  the  number  of 
smugglers  classified  and  captured.  This  simulation  study  found  there  was  an  increase  in  correct 
classifications  and  captures  with  the  aid  of  these  UAVs. 

Utilizing  similar  methodologies  as  the  works  above,  we  determine  the  benefit  or  penalty  to 
interdiction  efforts  by  the  addition  of  an  organic  UAV  capability  for  a  single  interdiction  team 
when  used  in  conjunction  with  a  surveyor  UAV.  We  measure  the  number  of  targets  captured  for 
a  given  AO  size  and  target  set.  Target  locations  are  not  be  known  a  priori  and  we  do  not  utilize  a 
predictive  algorithm  for  target  locations.  Specifically,  we  use  the  number  of  targets  captured  and 
the  time  to  capture  as  our  MOEs.  These  results  then  have  the  potential  to  affect  future  CONOPS 
for  the  integration  of  UAVs  with  ground  combat  elements.  Results  of  our  simulation  contribute 
to  the  development  of  a  decision  support  tool  that  can  aid  the  Commander  as  he  determines 
which  targets  to  interdict  based  on  which  presents  the  highest  priority. 

The  remainder  of  this  thesis  is  organized  as  follows.  In  Chapter  II,  presents  the  overall  scenario 
to  include  a  description  of  the  problem  domain.  In  this  chapter,  we  also  discuss  constraints, 
assumptions  and  present  the  simulation  model  formulation.  Chapter  III  discusses  the  exper¬ 
imental  design  used  and  presents  a  thorough  discussion  of  factors,  levels,  response  variables 
will  also  be  presented.  Chapter  IV  provides  the  analysis  of  the  design  and  any  insights  gained 
from  the  simulation.  Chapter  V  reports  the  conclusions  and  provide  any  recommendations  that 
can  be  incorporated  into  future  CONOPS  as  well  as  future  model  implementation. 
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CHAPTER  2: 
Model  Formulation 


This  chapter  will  discuss  the  SASIO  simulation  model  and  its  operational  context.  We  begin 
with  a  general  model  overview  followed  by  a  discussion  of  the  three  core  components:  the 
theater,  the  scenario,  and  the  task  force.  Each  component  presents  the  various  characteristics 
and  any  entities  it  possesses  and  discuss  the  motion  models  that  govern  the  entities. 

2.1  Operational  Context 

The  simulation  environment  possesses  an  Area  of  Interest  (AOI)  in  which  QRF  teams  are  lo¬ 
cated  at  a  FOB  with  the  mission  of  interdicting  and  capturing  hostile  targets.  Vehicle  mounted 
QRFs  will  be  restricted  to  the  existing  road  networks.  Foot  mobile  QRFs  are  not  restricted  to 
the  road  networks  and  although  terrain  often  dictates  ease  of  travel,  for  this  thesis,  terrain  will 
not  be  a  factor.  The  QRF  can  travel  either  by  foot  or  vehicle  (e.g.,  HMMWV)  to  the  target 
location.  The  QRF  will  remain  at  the  FOB  until  it  receives  a  mission. 

Airborne  assets  include  a  surveyor  UAV  that  will  perform  ISR  within  the  AOI.  This  UAV  will 
utilize  one  of  three  specified  search  patterns  to  determine  which  provides  the  most  target  de¬ 
tections.  Missions  will  be  generated  from  reports  sent  to  the  FOB  by  the  surveyor.  After  the 
report  is  generated  surveyor  can  perform  one  of  two  missions.  First,  it  can  continue  to  search 
for  additional  targets.  Alternatively,  it  can  transition  to  a  tracking  mode,  updating  target  action 
and  location  in  the  form  of  additional  reports  to  the  QRF.  Once  the  QRF  arrives  at  the  target 
location,  the  surveyor  will  be  released  back  to  its  search  mission. 

The  QRF  has  the  ability  to  have  a  tracker  UAV  assigned  to  it  as  an  organic  asset.  Faunched 
by  the  QRF  when  the  surveyor  issues  a  report,  the  tracker  will  proceed  to  the  last  known  target 
location.  If  the  surveyor  is  in  tracking  mode,  the  tracker  will  conduct  a  target  handoff,  once  it 
has  positively  identified  the  target,  releasing  the  surveyor  to  its  search  mission. 

2.2  Overview  of  the  Model 

SASIO  is  an  agent  based  simulation  model  written  in  the  Java  programming  language.  Agent 
based  modeling  is  used  to  study  the  interactions  of  autonomous  agents  in  complex  systems. 
These  models  are  used  to  simulate  the  actions  of  multiple  agents  in  an  attempt  to  predict  or 
re-create  complex  phenomena.  Monte  Carlo  techniques  can  be  used  to  introduce  randomness 
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to  the  model.  SASIO  defines  both  agent  and  object  modeling.  “Agents”  refer  to  friendly  forces 
and  “objects”  refer  to  either  neutral  or  hostile  targets.  SASIO  is  used  to  define  and  construct,  for 
example,  multiple  search  methods  for  agents,  agent  planning  and  interdiction  behaviors,  object 
motion  models,  and  their  respective  interactions  with  the  environment. 

2.3  Theater  Description 

Within  SASIO  the  theater  describes  the  region  in  which  all  entities  interact.  The  theater  for  this 
thesis  will  consist  of  a  range  of  AOIs  from  10  x  10  km  to  100  x  100  km.  Each  AOI  consists  of 
1  km  x  1  km  grid  squares  and  represent  any  location  where  real-world  operations  are  currently 
ongoing.  The  choice  for  this  size  grid  square  is  consistent  with  standard  unit  of  measure  for 
charts,  maps,  and  gridded  reference  graphics  used  by  ground  and  air  assets  in  the  current  theaters 
of  operation. 

Each  AOI  can  be  represented  by  an  undirected  graph  with  a  corresponding  symmetric  adjacency 
matrix,  which  defines  all  connections  in  the  network.  We  will  define  three  adjacency  matrices 
as  follows: 


Gi  is  a  grid  graph  where  C  =  100 
G2  is  a  grid  graph  where  C  =  2500 
G3  is  a  grid  graph  where  C  =  10,  000 


where  C  denotes  the  number  of  cells  within  each  AOI.  Each  graph  has  eight-point  connectivity, 
meaning  each  cell,  c,  may  be  connected  in  the  following  manner,  (NW,  N,  NE,  E,  SE,  S,  SW,  W} 
Edge  cells  are  connected  in  the  same  manner  but  with  fewer  adjacent  cells.  The  adjacency  ma¬ 
trix  takes  the  following  form: 


AdjGi 


110  0  0 
1110  0 
0  1110 
0  0  111 
0  0  0  1  1 


,  where  ay 


1  if  connected  by  eight  pt.  connectivity 
0,  otherwise 


These  AOIs  can  be  thought  of  as  abstractions  or  extensions  of  Camp  Roberts  in  California  (see 
Figure  2.1)  where  ongoing  research  and  experimentation  utilize  UAVs  and  ground  assets  to 
study  operationally  relevant  problems  such  as  proposed  in  this  thesis. 
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Figure  2.1:  Camp  Roberts  Gridded  Reference  Graphic 

2.4  Scenario  Description 

The  scenario  describes  the  number,  movement,  and  prediction  models  of  the  red  forces,  known 
as  objects.  Objects  in  this  thesis  can  either  be  targets  or  neutrals. 

2.4.1  Object  Placement  Model 

For  this  thesis,  we  will  consider  three  separate  object  sets.  We  will  let  N  be  the  number  of 
objects  and  N  e  {30,  75,  120}.  The  make  up  of  the  object  set  is  a  one  to  two  ratio  of  hostile 
to  neutral  objects.  For  example,  if  N  =  30,  10  objects  will  be  targets  and  20  will  be  neutral. 
Object  placement  is  assumed  independent  which  may  result  in  more  than  one  object  being 
placed  within  a  cell.  Each  object  set  will  be  placed  within  the  AOI  based  on  a  probability  map 
that  is  initialized  at  the  start  of  the  scenario  with  probabilities  of  an  object  being  placed  within  a 
cell.  We  will  define  the  probability  using  a  uniform  distribution  across  the  entire  theater  in  the 
following  manner: 

Let  the  discrete  random  variable  X=  the  location  of  an  object,  where  X  =  { 1,2,3, ...,C},  with 
probability  distribution: 
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P[ X  =  c]  =  the  probability  that  an  object  is  located  in  cell  c,  then  for  uniform  probabilities 


P\X  =  c]=l- 

In  order  to  generate  the  object  locations,  for  each  object  we  will  generate  a  random  number 
based  on  the  number  of  cells  in  the  AOI  and  then  place  the  object  at  that  cell  location.  For 
example, 


R~U(1,C) 

where  recall  that  C  is  the  number  of  cells  in  the  AOI.  This  algorithm  will  continue  until  all 
objects  have  been  assigned.  Since  all  object  locations  are  assumed  independent,  all  assignments 
will  follow  the  same  uniform  distribution. 

2.4.2  Object  Motion  Model 

In  order  to  model  object  motion,  we  need  to  know  the  object’s  current  location,  all  possible  fu¬ 
ture  locations,  and  the  time  step  utilized  to  measure  target  velocity.  Object  motions  are  modeled 
as  a  discrete  time  Markov  chain  with  a  time  step  equal  to  one  minute,  i.e.,  At  =  1.  A  discrete 
time  Markov  chain  is  specified  by  its  transition  matrix  and  its  initial  distribution.  The  objects’ 
initial  distribution,  as  described  above,  is  given  by 

P[X  —  c]  —  which  can  be  written  as  P[X i]  =  P[ X2]  =  P[X3\  =  ...  =  P[XC]  - 

Target  locations  can  be  determined  by  starting  with  their  initial  location  given  by  the  target 
placement  model.  The  theater  adjacency  matrix  represents  all  possible  future  locations  for  a 
target  in  a  given  cell.  The  adjacency  matrix  is  a  undirected  graph  that  depicts  all  possible  target 
flow  from  cell  to  cell.  When  the  future  locations  are  replaced  by  the  probability  of  an  object 
transitioning  to  that  cell,  the  resulting  matrix  becomes  the  theater  transition  matrix,  P.  In  order 
to  determine  the  transition  probabilities  we  use  the  following  algorithm.  This  algorithm  defines 
all  one  time  step  transition  probabilities. 

Tit+1=Pnt  ,  where  Tci+1  is  the  probability  of  transitioning  in  one  time  step,  P  is  the  transition 
matrix,  and  n*  is  the  current  location  probability.  This  thesis  will  explore  the  Random  Walk 
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motion  model  that  can  be  generalized  to  different  object  motions.  The  object  motions  will 
be  characterized  by  the  transition  probability  possessed  by  the  object  and  can  range  from  a 
stationary  target  to  a  target  that  transitions  at  every  time  step. 

Since  the  theater  is  represented  by  an  undirected  graph,  the  adjacency  matrix  is  symmetric 
meaning  if  i  is  connected  to  j,  then  j  is  connected  to  i.  A  row  of  an  adjacency  matrix  represents 
all  cells  adjacent  to  a  given  cell,  i.e,  if  a  cell  j  is  adjacent  to  cell  i ,  the  (i,j)—  entry  in  the 
adjacency  matrix  is  one.  The  sum  of  this  row  is  called  the  outdegree,  which  is  defined  as  the 
sum  of  all  arc  tailpoints  that  emanate  from  that  cell  and  connect  to  another  cell.  Formally 


c 

outdegree,  =  ^  ar<! 

j= i 


Non-Uniform  Random  Walk  Motion  Model 

In  the  non-uniform  random  walk  model,  the  probability  of  transitioning  to  any  adjacent  cell  is 
not  described  by  a  uniform  distribution.  Instead,  the  current  location  can  be  described  as  being 
self-important,  meaning  the  target  either  wants  to  quickly  leave  the  cell  in  which  it  is  located  or 
it  would  prefer  to  stay  at  the  current  location.  The  first  case  can  be  described  as  a  fast  random 
walk  and  the  second  as  a  slow  random  walk.  In  either  case,  this  self-importance  imparts  a  7 r.^, 
the  self-transition  probability  for  the  current  cell,  that  is  different  from  its  adjacent  cells  which 
is  the  cause  of  the  non-uniformity.  In  order  to  find  the  transition  probabilities  for  the  remaining 
cells  we  need  to  perform  the  following  calculation 


1 


outdegree  —  1 


,  j  ±  h  :j  e  Adj 


Gi 


nid  =  0  if  j  £  AdjGi 


Row  entries  for  each  c,  are  determined  in  this  manner.  The  resulting  matrix  is  the  transition 
matrix,  P. 

To  take  this  argument  one  step  further,  since  t is  a  continuous  factor,  we  see  that  as  tx iti 
approaches  the  limit  of  1,  the  transition  probability  to  other  cells  approaches  zero  and  thus  we 
have  the  special  case  of  the  non-uniform  random  walk  that  describes  a  stationary  target. 
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Figure  2.2:  Lawnmower  Search  Pattern 


It  can  also  be  shown  that  uniform  random  walk  is  a  special  case  of  the  non-uniform  random 
walk.  In  this  case  the  transition  probabilities  are  derived  by  the  following  equation 

1 

1,3  outdegree 

that  shows  the  object  is  equally  likely  to  go  to  all  adjacent  cells  and  therefore  has  a  transition 
matrix  that  is  uniform  for  each  discrete  time  step.  Uniform  walk  will  not  be  explicitly  studied, 
since  it  falls  within  the  continuum  defined  by  the  fast  and  slow  random  walk  cases. 

2.5  Task  Force  Description 

The  final  part  of  SASIO  is  the  taskforce  that  describes  the  blue  force  assets  to  include  the 
surveyor,  QRF,  and  the  tracker  UAV  organic  to  the  QRF.  The  taskforce  describes  all  blue  force 
agent  capabilities  and  motion  models. 

2.5.1  Surveyor  UAV 

Search  Pattern 

The  surveyor  UAV  represents  a  Group  3  system  and  will  perform  ISR  in  order  to  detect  hostile 
targets  within  the  AOI.  It  will  fly  one  of  three  search  patterns  as  directed  by  the  operator.  The 
possible  patterns  are  lawnmower,  spiral,  and  random  walk  (See  Figure  2.2). 

Identification 

The  surveyor  is  modeled  with  imperfect  sensing  capabilities  and  is  the  only  Blue  Force  as¬ 
set  that  must  address  this  issue.  The  main  characteristics  of  sensing  are  p,  the  false  negative 
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identification  probability  and  7,  the  false  positive  identification  probability.  When  the  surveyor 
makes  a  detection  there  are  four  possible  probabilities  for  target  detection  based  on  the  sensing 
characteristics.  These  cases  are  (1)  the  probability  of  a  correct  negative  identification  (correctly 
observing  no  target  present),  I-7,  (2)  the  probability  of  a  false  positive  identification  (incor¬ 
rectly  observing  a  target  when  none  is  present),  7,  (3)  the  probability  of  a  missed  detection 
(incorrectly  observing  no  target  when  one  is  present),  p,  and  (4)  the  probability  of  correctly 
detecting  a  target,  1-p.  These  probabilities  can  be  expressed  using  the  following  derivation: 

Let  Xc  be  a  Bernoulli  random  variable  describing  target  presence  as  defined  below 


no  target  present 
at  least  one  target  present 


Let  Yc  be  a  Bernoulli  random  variable  describing  target  detection  as  defined  below 


We  can  then  say, 


target  not  detected 
target  detected 


I-7  Ap[yc  =  0|Xc  =  o] 

7  =  P[YC  =  l\xc  =  0] 

p  =  P  [Yc  =  0\XC  =  1] 
l-p±P[Yc  =  l\Xc  =  l] 


Surveyor  Tracking  Behavior 

The  surveyor  is  modeled  as  having  one  of  two  different  capabilities.  In  the  first  case,  the  sur¬ 
veyor  does  not  have  a  tracking  capability.  It  executes  its  assigned  search  pattern,  as  per  section 
2.5.1  and  upon  making  a  detection  issues  a  report  containing  the  time  and  location  of  the  detec¬ 
tion.  Since  it  has  no  tracking  capability,  the  surveyor  will  continue  its  assigned  search  pattern 
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Figure  2.3:  Surveyor  with  Tracking  Capability  Finite  State  Machine 


and  make  additional  reports  as  necessary.  In  the  second  case,  the  surveyor  does  possess  a  track¬ 
ing  capability.  Upon  detection  of  a  target,  it  will  issue  a  report  but  now  transition  into  a  tracking 
mode.  The  surveyor  will  continue  to  track  the  target  until  it  released  by  the  QRF  or  the  tracking 
UAV  at  which  time  it  will  return  to  its  search  mission. 

2.5.2  Interdiction  Team  (QRF) 

Upon  initialization  of  the  simulation,  the  QRF  is  in  a  stationary  mode,  placed  at  the  FOB  loca¬ 
tion,  and  is  listening  for  reports  from  the  surveyor.  The  interdiction  team  will  have  two  modes 
of  transit:  foot  mobile  and  vehicle  mounted.  At  each  time  step,  the  QRF  listens  for  a  report 
and  if  one  is  received  it  will  begin  its  transit  to  the  requested  or  goal  location.  Once  the  goal 
location  is  reached,  the  QRF  will  clear  the  target,  and  check  for  any  unserviced  reports  from  the 
surveyor.  If  there  are  unserviced  reports,  the  QRF  will  transit  to  the  new  goal  location.  If  there 
are  no  new  reports,  the  QRF  will  transit  back  to  the  FOB.  Once  at  the  FOB,  the  QRF  returns 
to  stationary  mode  and  begins  listening  for  new  unserviced  reports.  It  is  worth  noting  that  the 
interdiction  team  brings  two  factors  with  three  levels  each  to  the  simulation.  The  first  is  transit 
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Figure  2.4:  Interdiction  Team  Finite  State  Machine 


time,  which  is  a  function  of  the  mode  of  travel  and  the  second  is  clear  time,  which  will  take  on 
one  of  three  values  which  will  represent  a  delay  at  the  goal  location.  This  process  will  continue 
until  either  all  targets  are  interdicted  and  cleared  or  the  simulation  ends. 

The  QRF  may  also  have  the  added  capability  of  an  organic  tracking  UAV.  The  QRF  model 
would  now  need  to  launch  the  tracker  at  the  specified  time  or  distance  from  the  goal  location. 
Also,  release  of  the  surveyor  would  be  initiated  by  the  tracker  UAV  once  it  is  has  received 
target  handoff  from  the  surveyor.  Now  when  the  QRF  reaches  the  goal  location,  it  will  assume 
tracking  responsibilities  and  release  the  tracker  for  any  additional  unserviced  reports. 

2.5.3  Tracker  UAV 

The  tracker  UAV  represent  a  Group  1  system  and  will  be  initialized  in  a  stationary  mode  co¬ 
located  with  the  QRF  and  awaits  tasking  from  the  QRF.  The  tracker  UAV  represents  a  portable, 
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Figure  2.5:  Tracker  UAV  Finite  State  Machine 


hand-launched  asset  organic  to  the  QRF.  As  the  QRF  transits  to  the  goal  location,  it  will  launch 
the  tracker  at  one  of  three  specified  distances  from  the  goal  location.  These  distances  are  one, 
three,  and  five  cells  from  the  goal  location.  The  tracker  velocity  is  60  and  1 80  kilometers  per 
hour  which  corresponds  to  approximately  35  and  100  knots.  It  is  also  assumed  to  have  perfect 
sensing  capability.  Upon  receipt  of  tasking,  the  tracker  is  launched  by  the  QRF  to  begin  its 
transit  to  the  goal  location.  After  receiving  the  target  handoff  from  the  surveyor,  the  tracker  will 
release  the  surveyor  back  to  its  search  mission  and  assume  tracking  responsibilities  until  arrival 
of  the  QRF.  As  the  QRF  arrives  at  the  goal  location,  it  will  release  the  tracker  for  any  additional 
unserviced  reports  if  they  exist.  If  there  are  none,  then  the  tracker  returns  to  the  QRF. 
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CHAPTER  3: 
Experimental  Design 


We  use  experimental  design  as  the  basis  for  determining  the  significant  factors  for  the  presented 
SASIO  simulation  model.  Experimental  design  allows  for  estimating  the  effects  of  the  input 
variables  simultaneously  where  variation  of  the  factors  is  present.  Screening  experiments  allow 
the  experimenter  to  vary  factors  in  such  a  way  that  the  most  significant  factors  can  be  identified 
with  respect  to  the  response  variables  of  interest,  with  as  little  experimental  effort  as  possi¬ 
ble.  The  SASIO  software  encapsulating  the  model  is  designed  to  utilize  design  of  experiments 
(DOX)  to  study  the  aspects  of  UAVs’  surveillance  characteristics  in  conjunction  with  ground 
based  interdiction  teams  with  the  overall  goal  of  increasing  the  number  of  targets  captured. 

The  model  created  for  this  thesis  has  several  response  variables  of  interest.  The  primary  re¬ 
sponse  variable  is  the  number  of  targets  captured.  This  is  the  metric  by  which  we  measure 
the  benefit  or  penalty  of  the  different  teaming  strategies.  The  second  to  be  quantified  is  the 
percentage  of  time  the  surveyor  UAV  actually  spends  performing  the  search  mission. 

The  remainder  of  this  chapter  introduces  the  response  variables  of  interest  as  well  as  the  factors 
to  be  used.  A  thorough  explanation  of  the  particular  experimental  design  used  will  also  be 
discussed.  Finally,  this  chapter  concludes  with  the  analysis  of  the  output  from  the  simulation 
runs. 


3.1  Response  Variables 

There  are  many  factors  and  levels  that  will  vary  during  this  experiment.  The  model  will  be 
used  to  simulate  the  unpredictability  of  all  entities  in  the  AOI  and  thus  will  provide  insights 
that  demonstrate  which  factors  have  a  significant  effect  on  the  response  variables.  We  have  two 
response  variables  for  this  simulation  model  and  their  descriptions  follow  below. 

3.1.1  Percentage  of  Targets  Cleared 

This  is  the  primary  response  variable  that  any  commander  will  wish  to  know.  The  percent 
targets  captured  directly  relates  to  how  efficiently  assets  perform  for  a  given  teaming  strategy. 
The  higher  this  number,  the  more  successful  the  teaming  strategy.  Insights  can  be  gained  on 
which  QRF  mobility  type  is  preferred  and  if  it  is  worth  the  effort  to  launch  the  tracker  based  on 
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Factor 

Levels 

Type 

Description 

Team  Type 

Surveyor 

Categorical 

UAV  capabilities  that 
are  available  to  the  QRF 

S  urvey  or/Tracking 

Surveyor  with  Tracker 

Search  Pattern 

Random  Walk 

Categorical 

patterns  to  be  flown  by 
Surveyor  UAV  only 

Lawnmower 

Spiral 

Tracker  Launch 

[1,3,5] 

Nominal 

#  cells  from  goal  location 

Interdictor  Transit  Time 

[15,2,1] 

Nominal 

#  time  steps  to  traverse  1  cell 

Tracker  Speed 

[1,3] 

Nominal 

#  cells  traversed  /  time  step 

Surveyor  Gamma  (7) 

[0,  0.9] 

Nominal 

Constraint 

7  +  p  <  0.9 

Surveyor  Rho  (p) 

[0,  0.9] 

Nominal 

Search  Area 

[100,  2500,  10000] 

Nominal 

#  cells,  C  in  the  AOI 

Interdictor  Clear  Time 

[1,11,21] 

Nominal 

#  time  steps 

Number  of  Objects 

[30,  75,  120] 

Categorical 

1:2  target  to  neutral  ratio 

Object  Motion 

Slow  Random  Walk 

Nominal 

Dependent  on  self 
transition  probabilities,  77^ 

Fast  Random  Walk 

Nominal 

Table  3.1 :  Factors  and  Levels 


QRF  distance  to  the  target.  This  response  will  be  obtained  directly  from  the  interdictor  cleared 
list  stored  within  SASIO. 


3.1.2  Percentage  of  Time  Surveyor  UAV  Performs  Search 

This  secondary  response  relates  to  the  percent  targets  cleared.  Of  interest  is  how  augmented 
tracking  affects  this  response  when  the  surveyor  possesses  a  tracking  capability.  This  response 
will  be  obtained  by  subtracting  the  surveyors  tracking  time  from  the  total  simulation  time  and 
then  dividing  by  total  simulation  time.  The  purpose  behind  the  different  teaming  strategies  is  to 
identify  the  strategy  that  results  in  the  highest  percent  of  targets  captured  per  simulation  run. 


3.2  Factors  and  Levels 

The  following  factors  and  levels  to  be  investigated  in  this  thesis  are  listed  in  Table  3.1. 

Each  factor  represents  a  characteristic  of  the  entities  in  the  simulation  and  represents  a  particular 
value  the  factor  can  take  during  the  course  of  the  mission.  Each  factor  and  its  expected  impact 
to  the  results  follow. 
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3.2.1  Team  Type 

Description 

The  Team  Type  is  the  main  focus  of  this  thesis.  Team  Type  is  the  ultimate  decision  a  comman¬ 
der  will  need  to  make  in  order  to  achieve  his  objectives.  The  first  strategy  utilizes  a  surveyor 
UAV  only  with  no  tracking  capability.  The  second  strategy  represents  a  surveyor  UAV  that  has 
tracking  capabilities.  The  third  strategy  will  utilize  the  surveyor  but  it  will  be  augmented  with 
an  additional  tracking  UAV,  organic  to  the  QRF.  This  factor  can  take  on  three  discrete  levels  and 
is  therefore  categorical. 

Predicted  Impact 

We  predict  the  augmented  tracking  provided  by  the  tracker  UAV  will  allow  the  surveyor  to 
spend  more  time  in  its  primary  mission  of  reconnaissance.  As  a  result,  more  targets  should  be 
identified  and  captured  during  the  course  of  the  simulation.  Thus,  an  increase  in  the  primary 
and  secondary  response  is  expected. 

3.2.2  Search  Pattern 

Description 

Search  pattern  is  also  a  categorical  variable  and  will  have  three  levels:  lawnmower,  spiral  out, 
and  random  walk.  With  the  exception  of  the  random  walk,  each  represents  a  predefined  set  of 
waypoints  for  the  surveyor  UAV  to  travel  as  it  searches  for  targets.  The  search  patterns  will  not 
be  affected  by  intelligence  related  to  target  locations.  UAVs  will  fly  the  specific  pattern  until 
a  target  is  detected,  at  which  time  a  report  will  be  issued  and  the  surveyor  will  either  track  the 
target  until  released  by  the  QRF  or  it  will  continue  to  search  depending  on  its  capabilities  and 
the  current  strategy. 

Predicted  Impact 

The  search  patterns  we  will  use  make  a  difference  if  there  are  a  large  number  of  targets  randomly 
clustered  together  where  the  UAV  has  the  ability  to  find  one  right  after  another.  If  the  targets 
are  placed  randomly  far  apart,  then  we  do  not  anticipate  the  search  pattern  having  much  of  an 
effect  on  the  percent  targets  captured. 

3.2.3  Tracker  Launch 

Description 

The  launch  of  the  tracker  UAV  is  modeled  using  three  thresholds,  representing  the  number  of 
cells  from  the  target  when  the  tracker  is  launched.  The  three  values  are  one,  three,  and  five  cells. 


21 


Predicted  Impact 

This  factor  when  combined  with  the  tracker  speed  will  effect  the  amount  of  time  the  surveyor 
will  spend  conducting  its  primary  mission  of  reconnaissance.  The  earlier  the  tracker  can  be 
launched  and  the  faster  it  can  arrive  at  the  goal  location,  the  sooner  it  can  relieve  the  surveyor 
and  thus  return  to  its  search  mission. 

3.2.4  Interdictor  Transit  Time 

Description 

These  values  will  represent  the  number  of  time  steps  required  to  traverse  from  one  cell  to  an 
adjacent  cell.  Each  value  simulates  different  modes  of  travel  from  a  foot  mobile  QRF  to  a 
vehicle  mounted  QRF  traveling  on  a  road  network  with  velocities  ranging  from  4kph  to  60kph. 

Predicted  Impact 

As  the  velocity  of  the  QRF  decreases  more  time  will  be  required  to  transit  to  a  goal  location. 
This  translates  into  more  time  the  surveyor  has  to  remain  on  station  before  being  released  by 
either  the  QRF  or  the  tracker,  which  we  predict  will  negatively  effect  the  number  of  targets 
identified  and  captured. 

3.2.5  Tracker  Speed 

Description 

This  factor  represents  the  different  velocities  the  tracker  UAV  may  fly.  We  assume  perfect 
sensing  for  the  tracker  so  speed  will  not  affect  tracker  7  and  p.  It  has  two  levels,  one  and  three, 
which  represent  the  number  of  time  steps  required  to  traverse  from  one  cell  to  the  next  cell  en 
route  to  the  target  location.  These  levels  are  equivalent  to  60  and  180kph. 

Predicted  Impact 

Assuming  surveyor  will  be  released  by  the  tracker,  the  quicker  the  tracker  can  arrive  at  the  goal 
location  to  release  the  surveyor,  the  sooner  the  surveyor  can  return  to  its  mission.  This  should 
lead  to  an  increase  in  the  percent  targets  captured  and  the  total  number  of  targets  identified. 

3.2.6  Sensor  Characteristics 

Description 

Two  continuous  factors  are  used  to  represent  the  imperfect  sensing  capabilities  of  the  surveyor 
UAV.  7  (gamma)  is  the  probability  of  having  a  false  positive,  which  is  identifying  a  target  as 
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hostile  when  it  is  not.  p  (rho)  is  the  probability  of  a  false  negative,  which  is  classifying  a  target 
as  friendly  when  it  is  in  fact  hostile. 

Predicted  Impact 

While  both  7  and  p  can  have  a  significant  effect,  7  is  a  greater  concern  due  to  the  misclassifica- 
tion  of  a  neutral  as  a  target.  The  higher  this  misclassification  rate,  the  more  time  the  QRF  will 
spend  clearing  objects  that  are  not  targets  and  thus  as  7  increases  for  the  surveyor,  we  would 
expect  to  see  the  percent  targets  captured  decrease. 

3.2.7  Search  Area 

Description 

The  search  area  is  a  continuous  factor  based  on  AOIs  composed  of  1  km  x  1  km  grid  squares. 
Varying  this  factor  will  also  help  quantify  the  benefit  or  penalty  of  the  QRF  having  an  organic 
UAV  tracking  capability. 

Predicted  Impact 

An  increase  in  search  area  should  make  it  more  difficult  to  locate  targets.  Therefore,  we  would 
expect  to  see  a  decrease  in  the  percent  targets  captured  as  the  size  of  the  search  area  is  increased. 

3.2.8  Clear  Time 

Description 

This  factor  represents  the  amount  of  time  it  takes  for  the  QRF  to  actually  interdict  and  capture 
a  target.  This  factor  takes  on  three  levels  as  shown  in  Figure  3.2. 

Predicted  Impact 

We  expect  that  as  the  clearing  time  is  increased  fewer  additional  targets  found  by  the  surveyor 
will  be  interdicted  and  cleared  by  the  QRF. 

3.2.9  Number  of  Objects 

Description 

The  number  of  objects  in  the  AOI  will  take  on  three  distinct  values.  The  objects  will  be  ran¬ 
domly  placed  throughout  the  AOI  and  will  have  no  affect  on  the  search  pattern  used  by  the 
surveyor  or  mode  of  travel  of  the  QRF.  The  particular  level  represents  a  1:2  ratio  of  targets  to 
neutrals  in  the  AOI.  For  example,  if  the  number  of  objects  equals  30,  there  are  10  targets  and  20 
neutrals  in  that  target  set. 
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Predicted  Impact 

As  this  factor  is  varied  with  different  AOI  sizes,  we  expect  to  see  significant  variation  to  the 
number  of  targets  identified  and  therefore  the  percent  targets  captured. 


3.3  Experimental  Design 

The  choice  of  experimental  design  was  developed  to  accommodate  eleven  input  factors.  This 
experiment  constitutes  a  mixed-level  design,  for  which  the  design  matrix  was  created  using  the 
ZJ-optimality  criterion  that  results  in  a  nearly  orthogonal  design.  The  result  is  nearly  orthogonal 
because  some  correlation  exists  among  the  columns  of  the  information  matrix  [X'X\.  A  D- 
optimal  design  attempts  to  minimize  the  variance  of  the  model  regression  coefficients  [9]. 

Factor  screening  is  the  process  of  systematically  varying  input  factors  in  order  to  identify  the 
factors  that  produce  a  significant  change  in  the  response  variables.  The  screening  experiment 
is  used  to  estimate  the  magnitude  and  direction  of  individual  factor  effects  as  well  as  factor  in¬ 
teraction  effects  on  the  response  variable.  As  previously  mentioned,  we  are  using  a  /9-optimal 
design  for  the  screening  experiment  in  an  attempt  to  minimize  the  variance  of  the  model  regres¬ 
sion  coefficients.  The  D-optimal  design  minimizes  the  volume  of  the  joint  confidence  region 
on  the  vector  of  regression  coefficients  by  minimizing  |  (X'X)~ 1 1.  which  equates  to  minimizing 
the  uncertainty  of  the  regression  coefficients. 

Traditional  screening  experiments  are  conducted  using  only  two  levels  of  the  factors,  a  low 
level  and  a  high  level.  Center  points  are  added  to  the  design  to  check  for  non-linearity  within 
the  model  and  to  reduce  the  variance  in  the  center  of  the  design  space.  The  resulting  design 
matrix  is  read  into  the  SASIO  model  and  the  simulation  conducted  using  the  actual  values  as 
found  in  Table  3.1.  The  design  matrix  developed  in  JMP  8.0.1  [10]  resulted  in  102  design  points 
including  six  center  points. 

In  order  to  analyze  the  data,  the  actual  values  in  Table  3.1  must  be  transformed  to  coded  values 
that  strips  the  units  from  any  level  and  turns  the  result  into  a  level  that  is  dimensionless.  This 
allows  the  magnitude  of  the  model  coefficients  to  be  directly  comparable  and  the  effect  of 
changing  each  design  factor  over  a  single  unit  is  easily  measured.  Coded  variables  are  effective 
for  determining  the  relative  size  of  factor  effects  and  allow  the  experimenter  to  see  the  relative 
importance  of  the  design  factors.  Output  data  is  then  analyzed  in  JMP  but  with  the  factor 
values  converted  to  coded  units  of  -1,  0,  and  1  to  represent  the  low,  center  point,  and  high 
levels  respectively.  There  were  some  cases  where  we  chose  a  third  level  that  was  not  the  exact 
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Std  Dev 


Figure  3.1 :  Asymptotic  Standard  Deviation 


center  point.  This  was  done  to  more  accurately  represent  the  level  as  a  real  world  value  for  the 
particular  level.  Actual  values  were  converted  to  coded  units  using  the  following  formula: 


coded  value 


Actual  value  —  (ActualLow  +  ActualHigh)/2 
(ActualHigh  -  ActualLow)/2 


3.4  Methods  of  Analysis 

3.4.1  Asymptotic  Variance 

Through  a  study  of  the  asymptotic  variance,  we  determined  60  replications  of  each  design  point 
was  adequate  to  gain  insights  from  the  simulation  model,  see  Figure  3.1.  We  ran  the  model 
for  one  design  point  multiple  times  while  varying  the  number  of  replications  until  the  standard 
deviation  of  both  response  varibles  stabilized.  This  occured  at  approximately  60  replications. 
We  performed  this  analysis  in  order  to  determine  the  minimum  number  of  runs  per  design  point 
while  being  able  to  gain  insights  from  the  simulation  with  the  goal  of  running  the  simulation  on 
a  tactical  laptop  in  the  field.  The  simulation  can  run  from  less  than  an  hour  to  approximately 
six  hours  depending  on  the  number  of  design  points. 
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3.4.2  Multivariate  Linear  Regression 

Multivariate  linear  regression  can  be  used  to  determine  what  factors  in  the  screening  experiment 
have  a  significant  effect  on  the  response.  In  a  linear  regression  model  the  response  variable,  y 
is  related  to  predictor  variables,  x  through  the  following  relationship 


y  —  A)  +  PlX\  +  /3 2X2  +  Pl,2xlx2  +  •  •  •  +  Pnxn  +  £ 

The  standard  multivariate  linear  regression  model  tests  the  following  hypothesis: 

H0  :  /30  =  [3ixx  =  /?i,2^i,2  =  0nxn  =  0,  where  nis  the  number  of  coefficients 
Hi  :  at  least  one  coefficient  ^  0 

3.4.3  Logistic  Regression 

Logistic  regression  is  used  for  the  prediction  of  the  probability  of  the  occurrence  of  an  event. 
The  event  we  are  interested  in  is  the  probability  of  a  target  being  cleared.  The  number  of  targets 
cleared  is  a  Bernoulli  random  variable  that  can  take  on  the  values  of  zero  or  one.  Either  the 
target  is  cleared  or  it  is  not.  When  a  response  variable  is  binary,  the  resulting  shape  of  the 
response  function  is  nonlinear  [11].  This  nonlinear  function  takes  the  form 

logit  =  x+cxp^a./^) 

An  examination  of  this  function  shows  that  it  is  easily  linearized  by  using  the  logit  transforma¬ 
tion  defined  below  as 

r /  =  In  ( 1 ,  where  r/is  the  linear  predictor  of  the  response  variable  y 

By  employing  this  transformation  we  are  able  to  linearize  the  response  variable  and  perform 
standard  multivariate  linear  regression. 

In  order  to  determine  the  primary  response  variable,  the  mean  number  of  targets  was  determined 
for  each  design  point  and  then  divided  by  the  number  of  targets  for  that  design  point  to  attain  the 
percent  targets  cleared.  The  resulting  percent  was  transformed  via  the  logit  function  to  ensure 
the  response  variable  values  remained  between  zero  and  one  and  is  shown  below 

logit(percent  targets  cleared)  =ln  (  p^ent  targets  cleared  \  ^ 

°  vr^  °  7  V  1  -percent  targets  cleared  J  1 
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CHAPTER  4: 
Numerical  Analysis 


As  mentioned  in  Chapter  3,  screening  experiments  were  performed  to  determine  the  significant 
factors  that  affect  the  two  response  variables:  (1)  percentage  of  targets  cleared,  and  (2)  the 
percentage  of  time  the  surveyor  UAV  performs  its  primary  mission  of  search. 

The  first  part  of  the  analysis  performed  was  to  determine  the  effect  of  Tracker  Launch  and 
Tracker  Speed  only  for  the  Team  Type  of  Surveyor  with  Tracker.  To  accomplish  this,  we  broke 
down  Surveyor  with  Tracker  into  five  separate  factors  each  representing  the  high-low  combi¬ 
nations  and  centerpoints  of  Tracker  Launch  and  Tracker  Speed.  This  was  done  to  isolate  the 
effects  and  because  these  effects  are  not  present  for  the  Surveyor  only  and  Surveyor/Tracking 
Team  Types.  Table  4.1  shows  how  we  broke  down  Surveyor  with  Tracker. 

The  analysis  was  performed  omitting  Tracker  Launch  and  Tracker  Speed.  Performing  the  anal¬ 
ysis  in  this  way  allowed  for  effects  of  Tracker  Launch  and  Tracker  Speed  to  be  preserved  for 
Surveyor  with  Tracker  while  at  the  same  time  being  removed  from  the  analysis  in  order  to  en¬ 
sure  those  factors  do  not  impact  the  Team  Types  where  the  Tracker  is  not  part  of  the  simulation 
model.  The  remainder  of  this  chapter  will  discuss  the  results  of  the  screening  experiments. 

4.1  Percentage  of  Targets  Cleared 

The  initial  regression  analysis  showed  eight  main  and  interaction  effects  were  significant.  Fur¬ 
ther  analysis  showed  that  an  equally  good  model,  in  terms  of  R 2,  could  be  achieved  with  only 
five  main  and  interaction  effects.  The  results  are  summarized  in  Figure  4.1. 


Team  Type 

Tracker  Launch  Level 

Tracker  Speed  Level 

Surveyor  with  Tracker  A 

1 

1 

Surveyor  with  Tracker  B 

-1 

-1 

Surveyor  with  Tracker  C 

1 

-1 

Surveyor  with  Tracker  D 

-1 

1 

Surveyor  with  Tracker  E 

0 

0 

Table  4.1 :  New  Team  Type-Surveyor  with  Tracker  A-E 
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T  [»  Response  LOGIT  Percent  Mean  Tgts  Cleared 
▼  Whole  Model 


Actual  by  Predicted  Plot 


Predicted  P<.0001  RSq-0.91 
RM5E=0.9772 


▼  Summary  of  Fit 

RSquare  0.914576 

RSquare  Adj  0.906034 

Root  Mean  Square  Error  0.977221 

Mean  of  Response  -4.21499 

Observations  (or  Sum  Wgts)  100 


Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

9 

920.1744 

102.242 

107.0637 

Error 

90 

85.9464 

0.955 

Prob  >  F 

C.  Total 

99 

1006.1208 

<.0001* 

Figure  4.1 :  Linear  Regression  model  of  LOGIT  transformation  of  Percent  Targets  Cleared 


Figure  4.2  lists  the  parameter  estimates  in  order  of  lowest  to  highest  p- value.  The  regressor  with 
the  largest  coefficient  is  Search  Area.  As  the  search  area  is  increased,  a  negative  response  in 
the  percent  of  targets  cleared  is  anticipated.  Since  our  analysis  was  conducted  using  coded  vari¬ 
ables,  we  can  obtain  the  odds  ratio  for  a  single  unit  increase  in  search  area  to  be  e~  256  =  0.077, 
not  taking  into  account  the  effects  of  the  other  factors.  This  odds  ratio  can  be  interpreted  as  the 
estimated  increase  or  decrease  in  the  Percentage  of  Targets  Cleared  [11]  per  unit  increase  on 
Search  Area.  Therefore  as  Search  Area  increases  from  its  mid  level  to  high  level,  the  odds  of 
success  is  approximately  8  out  of  100.  This  model  also  shows  that  utilizing  a  team  type  com¬ 
prising  only  a  Surveyor  asset  with  no  tracking  capability  produces  the  least  desireable  results 
in  terms  of  the  response.  The  corresponding  odds  ratio  is  e-2  37  =  0.09,  which  equates  to  a 
reduction  in  the  odds  of  success  to  9  out  of  100.  A  similar  argument  can  be  made  for  Interdictor 
Transit  Time  and  False  Negative  Probability  of  detection. 

The  interaction  of  Interdictor  Transit  Time  and  Number  of  Objects  is  worth  examining  even 
though  its  effect  is  relatively  small.  Figure  4.3  shows  that  with  a  large  number  of  objects, 
increasing  the  interdictor  transit  time  still  does  not  improve  the  percentage  of  targets  cleared. 
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Sorted  Parameter  Estimates 
Term 

Search  Area 

New  Team  Type[Surveyor] 

Interdictor  Transit  Time 
False  Neg  Prob 

New  Team  Type[Surveyor  with  Tracker  B] 
Interdictor  Transit  Time’Number  of  Objects 
New  Team  Type[Surveyor  with  Tracker  D) 
New  Team  Type[Surveyor  with  Tracker  C] 
New  Team  Type[Surveyor  with  Tracker  A) 


Estimate 

-2.566751 

-2.371272 

-0.744327 

-0.715261 

0.9453771 

-0.274924 

0.7168729 

0.3268827 

0.0389965 


Std  Error 

0.100065 

0.184337 

0.101361 

0.106146 

0.309938 

0.100843 

0.311846 

0.309746 

0.310307 


t  Ratio 


Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0030* 

0.0077* 

0.0238* 

0.2941 

0.9003 


Figure  4.2:  Sorted  Parameter  Estimates  of  LOGIT  Transformation  of  Percent  Targets  Cleared  in  Order  of  Signifi¬ 
cance 
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Figure  4.3:  Interaction  Profile  showing  the  change  in  response  as  Search  Area  size  interacts  with  New  Team  Type 


The  QRF  is  simply  overwhelmed  by  the  number  of  objects  to  interdict.  This  result  supports  the 
argument  for  more  QRF  or  patrols  if  a  target  rich  environment  is  anticipated. 

Tukeys  Least  Squares  Means  Differences  allows  us  to  perform  a  multiple  comparison  of  the 
means  to  determine  if  there  is  a  significant  difference  among  any  pair  of  factors.  Tukeys  method 
then  groups  the  mean  responses  that  are  not  statistically  different  as  seen  in  the  crossletter  report 
(Figure  4.4).  Also  shown  is  a  plot  showing  the  difference  in  the  means  between  Team  Types 
with  a  tracking  capability  and  those  without.  We  can  see  a  significantly  lower  mean  response  for 
Surveyor  when  compared  with  Surveyor/Tracking  and  Surveyor  with  Tracking.  This  supports 
our  conclusion  that  possessing  a  tracking  capability  will  positively  impact  the  response. 
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Least 


Level 

Surveyor  with  Tracker  B  A 
Surveyor  with  Tracker  D  A 
Surveyor/Tracking  A 
Surveyor  with  Tracker  C  A 
Surveyor  with  Tracker  A  A 
Surveyor  B 


Sq  Mean 

-2.742302 

-2.970806 

-3.344536 

-3.360797 

-3.648683 

-6.058951 


Levels  not  connected  by  same  letter  are  significantly 
different. 


f  LS  Means  Plot 


Figure  4.4:  Tukeys  Crossletter  Report  and  Plot  of  the  Least  Squares  Means  showing  New  Team  Type  Grouped  by 
Similar  Means 


4.1.1  Percentage  of  Targets  Captured  with  Tracking  Capability 

After  establishing  that  possessing  a  tracking  capability  is  better  than  no  tracking  capability,  we 
were  interested  in  the  significant  effects  when  the  Surveyor  Team  Type  was  not  included  in 
the  model.  For  this  analysis,  all  Surveyor  data  was  removed.  Initial  results  of  the  stepwise 
regression  produced  a  good  fit  with  17  main  and  interaction  effects.  Further  analysis  illustrates 
that  seven  main  and  interaction  effects  is  also  satisfactory.  Results  are  shown  in  Figures  4.5  and 
4.6. 

Figure  4.6  lists  the  significant  factors  from  most  to  least  influential  for  this  model.  Search  Area 
has  the  greatest  effect  on  the  response.  The  odds  ratio  is  e~2'29  =  0.10,  which  equates  to  a 
reduction  in  the  odds  of  success  to  10  out  of  100  per  unit  increase  of  Search  Area,  which  is  very 
close  that  of  the  model  with  Team  Type  Surveyor  included. 

Interdictor  Transit  Time  is  the  next  factor  in  the  list.  This  was  an  anticipated  result  as  was 
the  effect  of  the  surveyor’s  sensor  characteristics.  Increasing  the  False  Negative  Probability, 
p,  reduces  the  percentage  of  targets  captured  due  to  the  increase  in  targets  being  misclassified 
as  neutrals.  An  increase  in  the  Number  of  Objects,  coupled  with  other  factors,  increases  the 
response  simply  because  as  the  number  of  objects  increases  so  does  the  number  of  targets. 


30 


T  [▼  Response  LOGIT  Percent  Mean  Tgts  Cleared 
T  Whole  Model 
▼  Actual  by  Predicted  Plot 


Predicted  P<.0001  RSq -0.91 
RMSE=0.8291 


▼  Summary  of  Fit 

RSquare  0.913089 

RSquare  Adj  0.901304 

Root  Mean  Square  Error  0.829083 

Mean  of  Response  -3.29737 

Observations  (or  Sum  Wgts)  68 


Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

8 

426.07409 

53.2593 

77.4817 

Error 

59 

40.55533 

0.6874 

Prob  >  F 

C.  Total 

67 

466.62942 

<.0001* 

Figure  4.5:  Linear  Regression  Model  of  LOGIT  Transformation  of  Percent  Targets  Captured 


Sorted  Parameter  Estimates 

Term 

Search  Area 

Interdictor  Transit  Time 
False  Neg  Prob 

Search  Area-Number  of  Objects 
False  Neg  Prob-Number  of  Objects 
Search  Pattern!  Random  Walk]*False  Neg  Prob 
Search  Pattern!  Lawnmower]*False  Neg  Prob 
False  Pos  Prob 


Estimate 

Std  Error 

-2.296026 

0.106584 

-0.839092 

0.106551 

-0.786552 

0.128978 

0.4054962 

0.114252 

0.3097635 

0.108559 

-0.400553 

0.150251 

0.3800016 

0.158974 

-0.292024 

0.129502 

t  Ratio 


Prob>|t| 

<.0001- 

<.0001- 

c.0001- 

0.0008* 

0.0060- 

0.0099- 

0.0200- 

0.0279- 


Figure  4.6:  Sorted  Parameter  Estimates  of  LOGIT  Transformation  of  Percent  Targets  Cleared  in  Order  of  Signifi¬ 
cance 


Of  greater  interest  is  the  lack  of  any  effect  from  either  Tracker  Launch  or  Tracker  Speed.  In 
neither  model  do  these  factors  have  an  effect.  Tracker  capabilities  were  modeled  using  the 
Raven  UAV,  the  USMC’s  Group  I UAV.  Further  study  is  required  to  determine  if  a  Tracker  with 
greater  capabilities  would  have  a  greater  effect  on  the  response  variable. 


4.2  Percentage  of  Time  Surveyor  UAV  Performs  Search 

Performing  an  initial  analysis  of  search  time  yields  a  model  with  10  main  and  interaction  effects. 
Further  analysis  yields  a  model  with  only  four  main  and  interaction  effects  and  R2  of  0.92  and 
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T  [^Response  LOGIT  Percent  Search  Time 
T  Whole  Model 


-6  -4  -2  0  2  4  6  8  10  12 

LOGIT  Percent  Search  Time  Predicted 
Pc.0001  RSq  =  0.92  RMSE=1.2791 


Summary  of  Fit 


RSquare 

0.924418 

RSquare  Adj 

0.913993 

Root  Mean  Square  Error  1.279075 

Mean  of  Response 

4.088854 

Observations  (or  Sum  Wgts) 

100 

Analysis  of  Variance 

Sum  of 

Source  DF 

Squares 

Mean  Square 

F  Ratio 

Model  12 

1740.8602 

14S.072 

88.6728 

Error  87 

142.3349 

1.636 

Prob  >  F 

C.  Total  99 

1883.1951 

<.0001* 

Figure  4.7:  Linear  Regression  Model  of  LOGIT  transformation  of  Percent  Time  Search 


Sorted  Parameter  Estimates 
Term 

New  Team  Type(Surveyor) 

Search  Area 

New  Team  Type[Surveyor]*Search  Area 
False  Neg  Prob 

New  Team  Type[Surveyor  with  Tracker  C] 

New  Team  Type[Surveyor  with  Tracker  B] 

New  Team  Type[Surveyor  with  Tracker  D] 

New  Team  Type[Surveyor  with  Tracker  A)*Search  Area 
New  Team  Type[Surveyor  with  Tracker  Bl'Search  Area 
New  Team  Type[Surveyor  with  Tracker  A] 

New  Team  Type[Surveyor  with  Tracker  D]*Search  Area 
New  Team  Type[Surveyor  with  Tracker  CJ'Search  Area 


Estimate 

Std  Error 

6.3044172 

0.241679 

1.7617465 

0.162107 

-1.88394 

0.245388 

0.9745043 

0.143901 

-2.037017 

0.414664 

-1.340732 

0.405956 

-1.133776 

0.403812 

0.527073 

0.414751 

0.4872861 

0.409871 

-0.330549 

0.414664 

0.2000334 

0.409376 

0.196679 

0.414881 

t  Ratio 


Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0014* 

0.0062* 

0.2072 

0.2377 

0.4275 

0.6263 

0.6366 


Figure  4.8:  Sorted  Parameter  Estimates  of  LOGIT  transformation  of  Percent  Time  Search  in  order  of  significance 


adjusted  R 2  of  0.91.  The  results  are  shown  in  Figure  4.7. 

The  results  shown  in  Figures  4.7  and  4.8  show  that  we  would  reject  the  null  hypothesis  in  favor 
of  the  alternative  at  any  reasonable  confidence  level. 

When  all  Team  Types  are  included  in  the  model,  the  predictive  power  of  this  model  can  be 
skewed  by  the  Surveyor  team  type  as  shown  in  Figure  4.8.  This  is  due  to  Surveyor  having  no 
tracking  capability,  which  therefore  ensures  that  it  will  spend  the  entire  duration  of  the  mission 
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T  &  Response  LOGIT  Percent  Search  Time 
▼  Whole  Model 

Actual  by  Predicted  Plot 


▼  Summary  of  Fit 

RSquare  0.92212 

RSquare  Adj  0.915839 

Root  Mean  Square  Error  0.818547 

Mean  of  Response  1.427759 

Observations  (or  Sum  Wgts)  68 


Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

5 

491.85541 

98.3711 

146.8185 

Error 

62 

41.54114 

0.6700 

Prob  >  F 

C.  Total 

67 

533.39655 

<.0001* 

Figure  4.9:  Linear  Regression  Model  of  LOGIT  Percent  Time  Search 


searching.  Of  additional  interest  is  how  Percent  Search  Time  responds  when  the  team  types 
include  a  tracking  capability,  which  is  described  in  the  following  section. 


4.2.1  Surveyor  Search  Time  Percentage  with  a  Tracking  Capability 

The  initial  analysis  excluding  the  Surveyor  only  Team  Type  yielded  a  model  with  eight  main 
and  interaction  effects.  Further  analysis  showed  that  a  model  with  only  five  factors  produced  an 
R2  of  0.92  and  an  adjusted  R 2  of  0.91.  These  results  are  summarized  in  Figure  4.9  and  4.10. 

The  sorted  parameter  estimates  in  this  case  show  that  the  greatest  effect  is  due  to  the  Search 
Area.  The  odds  ratio  is  e-2  34  =  0.09  which  is  a  reduction  in  the  odds  of  success  to  9  out  of 
100  for  each  unit  increase  in  Search  Area  size.  Next,  the  False  Negative  Probability  increases 
the  search  time  due  to  an  increase  of  the  misclassification  rate  of  targets  as  neutrals.  This  leads 
to  Surveyor  not  tracking  and  thus  increases  the  search  time  at  the  expense  of  targets  captured. 
The  next  factor  with  the  greatest  effect  is  the  Number  of  Objects.  As  the  Number  of  Objects 
increases,  the  percentage  of  time  the  Surveyor  searches  will  decrease  due  to  the  tracking  time 
involved  with  each  additional  target  found.  False  Positive  Probabihty  has  the  opposite  effect 
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Sorted  Parameter  Estimates 
Term 

Search  Area 

False  Neg  Prob 

Number  of  Objects 

Interdictor  Transit  Time'Search  Area 

Flase  Pos  Prob’False  Neg  Prob 


Estimate  Std  Error 

2.3492065  0.102521 

1.7504826  0.127077 

-0.707536  0.102929 

0.6905477  0.102882 

0.5635082  0.124617 


t  Ratio 


22.91 

13.77 

1 

-6.87 

E 

6.71 

4.52 

Prob>|t| 
<.000 1* 
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Figure  4.10:  Sorted  Parameter  Estimates  of  LOGIT  Percent  Time  Search  in  order  of  significance 
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Figure  4.1 1 :  Interaction  Plots  of  Search  without  Surveyor  Team  Type 


of  False  Negative  Probability.  As  the  False  Positive  Probability  rate  increases,  Surveyor  spends 
less  time  searching  and  more  tracking  neutral  targets. 

The  interaction  of  Interdictor  Transit  Time  and  Search  Area  provides  an  interesting  insight  to 
this  model  as  depicted  in  Figure  4.1 1. 

An  analysis  of  the  interactions  without  the  Surveyor  Team  Type  is  shown  in  Figure  4.11.  This 
plot  shows  that  while  Search  Area  is  at  its  high  level  (1),  as  the  Interdictor  Transit  Time  de¬ 
creases  (faster  interdictor  speed)  the  percentage  search  time  decreases.  At  an  Interdictor  Transit 
Time  of  15kph,  its  high  level,  the  Search  Area  is  so  large  that  the  Surveyor  spends  a  longer 
time  tracking  than  searching  while  the  QRF  is  en  route  to  the  target  location.  At  the  low  level 
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Search  Pattern 

Lawnmower 

Tracker  Launch 

3 

Interdictor  Transit  Time 

1 

Tracker  Speed 

3 

False  Positive  Probability 

0.3 

False  Negative  Probability 

0.3 

Clear  Time 

20 

Number  of  Objects 

120 

Object  Motion 

SlowRW 

Table  4.2:  Factor  Levels  used  for  Sensitivity  Analysis 


of  Search  Area,  as  Interdictor  Transit  Time  decreases,  the  percentage  search  time  actually  in¬ 
creases.  Due  to  the  small  size  of  the  AOI  at  the  low  level,  the  interdictor  is  able  to  get  to  the 
goal  location  very  quickly  releasing  the  surveyor  to  continue  its  search  mission  in  less  time  than 
in  a  larger  AOI. 

4.3  Sensitivity  Analysis 

Having  determined  that  Search  Area  and  Team  Type  have  the  largest  effect  on  both  response 
variables,  we  performed  sensitivity  analysis  to  quantify  those  effects.  All  factors  were  held 
constant  with  the  exception  of  Team  Type  and  Search  Area.  The  factor  values  used  are  shown 
in  Table  4.2 

Both  were  varied  across  their  respective  ranges  or  categories  that  resulted  in  eighteen  design 
points.  The  results  shown  in  Figure  4.12  clearly  indicate  that  having  a  tracking  capability  is 
better  than  no  tracking  capability.  Holding  all  factors  fixed  with  the  exception  of  Search  Area 
and  Team  Type  indicates  that  as  the  search  area  is  increased  the  percentage  of  targets  cleared 
decreases. 

Figure  4.13  also  shows  that  as  the  Search  area  increases  the  search  time  of  the  Surveyor  also 
increases. 

It  is  interesting  to  note  that  while  possessing  a  tracking  capability  is  superior  to  no  tracking 
capability,  there  is  no  statistical  difference  between  the  Team  Types  Surveyor/Tracking  and 
Surveyor  with  Tracker.  For  both  MOEs,  it  is  the  tracking  effect  that  affects  the  MOE,  not  the 
tracking  Team  Type.  As  mentioned  in  Section  4.2.1,  the  limiting  factor  for  these  Team  Types 
is  the  Interdictor  Transit  Time.  Regardless  of  how  the  tracking  capability  is  employed,  the 
number  of  targets  cleared  in  a  given  mission  time  is  limited  by  how  many  targets  the  interdictor 
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Percent  Targets  Cleared  vs.  Search  Area 

30.0 


Surveyor  Surveyor  with  Tracker  Surveyor/Tracking 

Team  Type 

Figure  4.12:  Average  Percent  Targets  Cleared  vs.  Search  Area  Size 


can  actually  get  to  during  the  mission. 


4.4  Analysis  Summary 

Based  on  the  screening  for  each  response,  in  all  cases  the  factor  with  the  greatest  effect  is  the 
Team  Type  and  Search  Area.  Surveyor  Team  Type  produces  a  low  response  for  the  Percentage 
of  Targets  cleared  due  to  the  lack  of  tracking  capability.  As  shown  in  Figure  4.13,  possessing 
a  tracking  capability  reduced  the  amount  of  time  the  Surveyor  searches  as  it  performs  its  sec¬ 
ondary  mission  of  track  rather  than  search.  Neither  of  these  conclusions  indicate  that  the  QRF 
having  an  organic  Tracker  UAV  is  a  valuable  asset.  The  sensitivity  analysis  performed  only 
on  the  Team  Types  with  a  tracking  capability  produced  similar  results.  For  these  team  types, 
Tracker  Launch  and  Tracker  Speed  produced  no  significant  effects  for  either  response  variable. 
In  order  to  produce  an  effect  in  either  response  variable  from  Tracker  Launch  and  Speed,  we 
would  recommend  performing  the  analysis  with  a  Tracker  UAV  of  greater  capability. 

We  have  also  discovered  that  the  hard  part  of  target  capture  is  not  the  interdiction  of  targets 
but  the  search  for  them  which  relates  directly  to  the  Search  Area  size.  How  we  search  and  the 
assets  available  continue  be  the  main  limiting  factors  in  target  acquisition.  The  number  of  team 
types  and  assets  available  is  an  area  for  future  study  to  help  determine  an  acceptable  size  AOI 
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for  mission  accomplishment.  For  the  models  above  we  also  found  Interdictor  Transit  Time  to 
be  a  limiting  factor  for  the  cases  where  we  studied  team  types  with  a  tracking  capability.  The 
QRF  can  only  clear  the  targets  it  can  get  to  during  the  course  of  a  mission  and  this  is  limited 
by  how  fast  it  can  get  to  each  target.  The  effect  of  increasing  or  decreasing  Interdictor  Transit 
Time  is  an  area  worthy  of  future  study.  If  it  makes  a  difference  in  terms  of  the  percentage  of 
targets  cleared,  then  an  analysis  of  alternatives  may  be  necessary  to  determine  the  best  form  of 
mobility  for  the  QRF  of  the  future. 
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CHAPTER  5: 

Conclusions  and  Recommendations 


We  set  out  to  answer  the  following  research  questions: 

•  Is  it  better  for  an  interdiction  team  to  possess  an  organic  “augmented  tracking”  capability 
in  the  context  of  time  to  intercept  and  number  of  targets  captured? 

•  What  are  the  significant  factors  that  produce  teaming  strategies  that  result  in  the  greatest 
number  of  hostile  targets  captured? 

The  analysis  also  shows  that  while  the  tracking  capability  is  better,  the  QRF  possessing  an  or¬ 
ganic  tracking  capability  had  no  effect  on  either  of  the  MOEs.  We  also  explored  three  Team 
Types  to  determine  which  factors  affect  the  success  of  those  Team  Types  in  terms  of  the  per¬ 
centage  of  targets  cleared  and  the  percentage  of  time  the  surveyor  searches.  Those  factors  are 
Team  Type  and  Search  Area  size. 

5.1  Search  Area 

It  was  not  surprising  to  leam  that  as  the  Search  Area  increased,  fewer  targets  were  captured. 
This  was  true  regardless  of  the  Team  Type  employed.  While  not  used  as  a  prediction  tool,  the 
results  of  this  simulation  provide  useful  insights  to  a  commander  who  is  planning  for  a  mission 
utilizing  teams  similar  to  the  ones  studied  here.  Based  on  the  given  AOI,  the  commander  may 
elect  to  employ  a  greater  number  of  teams  or  partition  the  AOI  into  smaller  sections  and  employ 
teams  in  each  partitioned  area. 

5.2  Team  Type 

Through  the  analysis,  we  were  able  to  show  that  possessing  a  tracking  capability  is  superior 
to  no  tracking  capability.  When  tracking  was  present  in  the  Team  Type,  the  number  of  targets 
cleared  for  a  given  mission  is  greater  than  when  tracking  was  absent.  Assuming  that  the  Sur¬ 
veyor  is  national  asset  and  there  may  only  be  one  available  for  the  AOI  and  it  may  be  unable  to 
track  until  released  by  the  QRF  or  Tracker,  the  team  types  employed  should  include  a  Tracker 
UAV.  This  would  allow  the  Tracker  to  be  launched  to  the  reported  target  location  and  get  “eyes 
on”  the  location  to  track  the  target.  If  the  target  is  not  found,  the  Tracker  can  then  perform  a 
search  as  the  QRF  approaches  in  an  attempt  to  reacquire  the  target. 
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5.3  Future  Works 

As  previously  mentioned,  an  area  for  future  research  is  the  effects  on  the  MOEs  of  multiple 
teams  in  a  given  AOI.  Specifically,  a  heuristic  can  be  developed  that  can  attempt  to  maximize 
the  number  of  targets  captured  by  minimizing  the  time  it  takes  for  an  interdiction  team  to  arrive 
at  a  target  location.  This  algorithm  can  solve  a  shortest  path  problem  to  the  target  from  each 
team,  identify  the  closest  team,  and  then  through  an  interface  with  SASIO  an  order  will  be  sent 
to  that  team  to  begin  the  interdiction  mission. 

We  showed  that  the  Interdictor  Transit  Time  was  a  limiting  factor  in  the  percentage  of  targets 
captured.  An  analysis  of  alternatives  can  also  be  conducted  to  determine  the  fastest  form  of 
mobility  for  a  QRF  based  on  current  or  future  systems.  A  reduction  in  the  transit  time  should 
lead  to  an  increase  in  the  percentage  of  targets  cleared  in  a  mission. 

Other  areas  where  the  SASIO  analysis  tool  would  be  useful  include  the  Navy’s  use  of  Unmanned 
Underwater  Vehicles  (UUV)  and  the  USMC  study  of  cargo  UAS  for  battlefield  resupply.  The 
Navy  envisions  the  UUV  to  have  shallow  water  capability,  stealth,  and  the  ability  conduct  ISR 
as  well  as  antisubmarine  warfare  and  mine  laying  operations.  The  cargo  UAS  problem  will  be 
affected  by  numerous  factors  including  range,  payload,  altitude  and  routes.  SASIO  provides 
the  ability  to  perform  a  thorough  analysis  of  the  factor  space  for  these  problems  and  determine 
which  factors  will  significantly  affect  the  UUV  and  cargo  UAS  operations.  Once  these  factors 
are  determined  a  better  CONOPS  can  developed  for  their  use.  While  these  systems  greatly 
enhance  surveillance  capabilities,  their  greatest  contribution  will  be  their  ability  to  aid  friendly 
forces  in  making  decisions  that  will  directly  lead  to  an  increase  in  the  number  of  hostile  targets 
captured.  This  is  one  of  the  overall  goals  of  these  systems.  Surveillance  is  simply  a  sub-problem 
of  the  surveillance  and  interdiction  mission  set. 
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APPENDIX  A: 

NPS-USSOCOM  Field  Experimentation 


A.l  General  Description 

Field  experiments  were  conducted  to  test  the  various  teaming  strategies  previously  described. 
The  experimentation  was  conducted  as  part  of  the  USSOCOM-NPS  Field  Experimentation  Co¬ 
operative  Capabilities  Based  Experiment  (CBE)  10-3  at  Camp  Roberts  Army  Reserve  Base. 

Assets  used  included  two  Raven  UAVs  to  simulate  the  Surveyor  and  Tracker  UAVs  as  well  as 
three  ground  vehicles  to  simulate  the  blue  team  QRF  and  two  red  team  target  vehicles.  The 
Raven  UAV  is  a  Group  one  UAV  according  to  the  Joint-UAS  definition.  It  has  a  wingspan  of 
4ft  3in  and  a  length  of  3ft  7in.  It  weighs  41bs  3oz  with  a  range  of  6.2  miles  or  roughly  10  km. 

Each  vehicle  was  equipped  with  a  radio  and  handheld  global  positioning  system  (GPS)  unit  to 
record  position  and  the  time  of  significant  events.  The  data  was  collected  at  the  TOC  and  stored 
locally  with  each  vehicle  for  post-experiment  playback  and  analysis. 

The  controllable  variables  for  these  experiments  included  Team  Type,  Search  Pattern,  and  the 
number  of  objects  with  the  understanding  that  we  would  not  be  able  to  control  the  number  of 
objects  in  any  AOI.  The  uncontrollable  variables  were  the  remainder  of  the  factors  in  Table  3.1. 
We  did  not  pick  any  particular  search  pattern  but  instead  performed  a  route  search  of  the  Camp 
Roberts  road  network.  This  decision  was  based  on  the  time  constraint  of  the  experiment. 

Due  to  asset  and  time  constraints,  the  experiment  was  originally  planned  to  study  two  teaming 
strategies  (Surveyor/Tracking  and  Surveyor  with  Tracker)  holding  all  other  factors  constant. 
An  operational  thread,  or  script,  was  provided  to  blue  and  red  teams  for  the  conduct  of  the 
experiment.  Multiple  experiments  were  conducted  during  the  allotted  time  period.  After  all  red 
teams  were  interdicted  or  the  FOB  was  attacked,  the  experiment  was  reset  for  the  next  iteration. 

Operational  Lessons  Learned 

The  results  of  these  field  experiments  validated  the  belief  that  the  search  effort  is  hard.  Locating 
hostile  objects  within  any  size  AOI  is  not  an  easy  task.  The  results  showed  that  even  for  a  small 
AOI  with  known  routes  of  travel  for  red  teams,  more  often  than  not,  the  red  teams  made  it  to 
the  FOB  before  being  interdicted  by  the  blue  QRF.  Detection  of  Red  Force  vehicles  by  the  Blue 
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Figure  1:  Raven  Launch 


Force  patrolling  UAV  (Raven)  was  infrequent,  limited  by  field-of-view,  unsteady  full-motion 
video,  and  scale  of  the  area  of  interest  (AOI).  In  some  cases  the  first  indication  of  the  presence 
of  the  red  team  came  from  the  QRF  at  the  FOB  when  the  red  team  came  within  line  of  sight 
of  the  FOB.  Once  the  QRF  made  contact  with  the  red  team,  either  from  Raven  cueing  or  self 
cueing,  the  actual  interdiction  effort  was  relatively  easy.  In  today’s  very  uncertain  environments, 
it  becomes  imperative  that  the  search  effort  be  successful.  Future  work  should  focus  not  only  on 
MUT  but  also  on  optimizing  the  search  effort  to  increase  the  effectiveness  of  the  MUT  effort. 

One  of  the  most  significant  and  uncontrollable  factors  in  any  tactical  operation  is  the  weather. 
These  experiments  validated  the  need  for  the  right  type  of  UAV  being  paired  with  the  appropri¬ 
ate  mission.  Due  to  equipment  constraints  one  of  the  Ravens  was  used  as  the  Surveyor  UAV. 
In  some  cases,  the  Surveyor  lost  track  due  to  head  winds  reducing  its  ground  speed  to  the  point 
where  it  was  no  longer  able  match  the  red  team  velocity  and  thus  the  red  team  drove  out  of 
the  Surveyor  field  of  view  and  contact  was  lost.  Constraints  in  real-world  settings  should  be 
considered  in  decision  support  models  for  employment. 

Weather  is  a  valid  consideration  at  the  tactical  level  also.  The  situation  encountered  above  may 
alter  the  decision  to  launch  a  Raven  as  a  tracking  asset  if  the  weather  might  make  it  impossible 
for  it  to  track.  In  these  situations,  the  Surveyor  would  be  required  to  perform  a  greater  share  of 
the  tracking  mission  and  thus,  we  would  expect  fewer  hostile  objects  to  be  found.  The  current 
SASIO  model  does  not  take  into  account  weather  effects.  Weather  and  its  effects  on  teaming 
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and  decisionmaking  is  an  area  that  should  be  considered  for  future  study.  CBE  10-4  scheduled 
for  August  2010  will  continue  these  efforts. 


A.2  Operational  Thread  Script 

Surveyor  /  Tracking 

T-0:  SURVEYOR  launches  on  a  broad  area  surveillance  mission  and  receives  tasking 

from  SASIO/Command  to  search  a  given  location  for  a  mounted  target  (RED  Team 
SUV) 

•  Intel  developed  and  passed  to  SURVEYOR  through  TOC 

•  SURVEYOR:  transitions  to  suspected  RED  Team  location 

•  QRF:  stationary  at  FOB  awaiting  tasking  from  TOC 

T- 1 :  SURVEYOR  locates  RED  Team  SUV 

•  SURVEYOR:  RED  Team  location  report  passed  to  TOC 

•  TOC:  updates  SASIO:Command 

•  QRF:  receives  tasking  from  TOCC  to  interdict  RED  Team  SUV  at  reported  loca¬ 
tion. 

T-2:  SURVEYOR  transitions  to  tracking  of  RED  Team  SUV 

•  SURVEYOR:  begins  overhead  orbit  of  RED  Team  SUV.  Reports  orbit  location 
via  SASIO:  Command 

•  TOC:  updates  SURVEYOR  position  in  SASIO:  Command 

•  QRF:  acknowledges  receipt  of  RED  Team  location  to  TOC.  QRF  begins  transition 
to  reported  RED  Team  location 

T-3:  QRF  arrives  at  reported  RED  Team  SUV  location 

•  QRF:  reports  arrival  at  RED  Team  suspected  location  and  reports  to  TOC 

•  QRF:  identifies  RED  Team  and  releases  SURVEYOR.  Makes  report  to  TOC 

•  SURVEYOR:  receives  new  tasking  from  TOC  via  SASIO:  Command 

•  QRF:  begins  clearing  of  RED  Team. 

T-4:  QRF  checks  for  new  tasking 

•  QRF:  reports  end  of  clearing  operations  to  TOC 

•  QRF:  request  additional  tasking  from  TOC 

T-5:  If  a  new  report  is  present,  QRF  transitions  to  new  RED  Team  SUV  location.  If  no 

new  report,  QRF  returns  to  FOB 

T-6:  After  all  RED  Team  targets  are  captured  or  at  the  end  of  the  experiment,  SUR¬ 

VEYOR  and  QRF  will  RTB 
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•  Collect  Data 

o  Number  of  RED  targets  reported 
o  Number  of  RED  targets  cleared 
o  Amount  of  time  SURVEYOR  searched 
o  Amount  of  time  SURVEYOR  conducted  other  tasks 

•  Collect  Video  of  Mission 

o  SURVEYOR  video  of  identified  RED  target 


Surveyor  with  Tracker 

T-0:  SURVEYOR  launches  on  a  broad  area  surveillance  mission  and  receives  tasking 

from  SASIO/Command  to  search  a  given  location  for  a  mounted  target  (RED  Team 
SUV) 

•  Intel  developed  and  passed  to  SURVEYOR  through  TOC 

•  SURVEYOR:  transitions  to  suspected  RED  Team  location 

•  QRF:  stationary  at  FOB  awaiting  tasking  from  TOC 

T- 1 :  SURVEYOR  locates  RED  Team  SUV 

•  SURVEYOR:  RED  Team  location  report  passed  to  TOC 

•  QRF:  receives  tasking  from  TOC  via  SASIO:  Command  to  interdict  RED  Team 
SUV  at  reported  location 

T-2:  SURVEYOR  transitions  to  tracking  of  RED  Team  SUV 

•  SURVEYOR:  begins  overhead  orbit  of  RED  Team  SUV.  Reports  orbit  location 

•  QRF:  acknowledges  receipt  of  RED  Team  location  to  TOCC.  QRF  begins  transi¬ 
tion  to  reported  RED  Team  location 

T-3:  QRF  begins  transition  to  reported  RED  Team  SUV  location 

•  QRF:  begins  transition  to  reported  RED  Team  location 

•  SURVEYOR:  continues  orbit  over  RED  Team.  Provides  video  feed  to  QRF  and 
ground  station 

•  TOCC:  updates  SASIO: Command  with  SURVEYOR  location  and  RED  Team 
activity 

T-4:  QRF  launches  TRACKER  at  predetermined  distance 

•  QRF:  launches  TRACKER  to  RED  Team  location.  Reports  launch  to  TOC 

•  SURVEYOR:  continues  overhead  orbit  and  providing  RED  Team  location  up¬ 
dates 

•  TRACKER:  begins  transit  to  RED  Team  location 
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T-5:  TRACKER  arrives  at  RED  Team  location 

•  SURVEYOR:  conducts  handoff  of  RED  Team  with  TRACKER 

•  TRACKER:  assumes  tracking  responsibilities  of  RED  Team 

•  TRACKER:  releases  SURVEYOR  for  additional  tasking 

•  SURVEYOR:  receives  new  tasking  from  TOC 

•  QRF:  continues  transit  to  RED  Team  location 

T-6:  QRF  arrives  at  reported  RED  Team  SUV  location 

•  QRF:  reports  arrival  at  RED  Team  suspected  location  and  reports  to  TOC 

•  QRF:  identifies  RED  Team  and  releases  SURVEYOR.  Makes  report  to  TOC 

•  TRACKER:  RTBs  to  QRF  •  QRF:  begins  clearing  of  RED  Team 

T-7:  QRF  checks  for  new  tasking 

•  QRF:  reports  end  of  clearing  operations  to  TOC 

•  QRF:  request  additional  tasking  from  TOC 

T-8:  If  a  new  report  is  present,  QRF  transitions  to  new  RED  Team  SUV  location.  If  no 

new  report,  QRF  returns  to  FOB 

T-9:  After  all  RED  Team  targets  are  captured  or  at  the  end  of  the  experiment,  SUR¬ 

VEYOR  and  QRF  will  RTB 

•  Collect  Data 

o  Number  of  RED  targets  reported 
o  Number  of  RED  targets  cleared 
o  Amount  of  time  SURVEYOR  searched 
o  Amount  of  time  SURVEYOR  conducted  other  tasks 

•  Collect  Video  of  Mission 

o  SURVEYOR  video  of  identified  RED  target 
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APPENDIX  B: 

Primer  on  Statistical  Analysis  Using  JMP 


Overview 

This  appendix  will  provide  the  reader  with  the  steps  necessary  to  construct  a  design  matrix  in 
JMP  [10]  and  perform  screening  analysis.  This  appendix  assumes  the  reader  has  access  to  a 
current  version  of  JMP  and  is  familiar  with  the  basic  statistical  operations. 


Step  1 

Open  JMP  and  select  DOE  from  the  menu  bar,  then  select  Custom  Design 


Figure  2:  Choose  DOE 


Step  2 

From  the  custom  design  dialog  window  under  Factors,  select  Add  Factors 


Figure  3:  Add  Factors 
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Add  all  factors  and  specify  their  type  or  role  and  the  associated  values.  Values  can  be  added 
as  coded  variables  for  analysis  or  the  engineering  units  can  be  added  at  the  high  and  low  levels 
for  input  into  a  simulation  model  such  as  SASIO.  Once  all  factors  and  levels  are  entered,  select 
Continue. 


Step  3 

Select  Interactions,  then  select  2nd  to  build  a  design  matrix  with  main  effects  and  1st  order 
interactions.  Then  select  Make  Design. 


Figure  4:  Interactions  selection 


After  interactions  are  selected  the  model  box  should  show  the  interactions 
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Figure  5:  Model  with  Interactions 


Step  4 

Before  selecting  Make  Table,  select  the  red  triangle  next  to  Custom  Design,  select  Optimality 
Criterion,  select  appropriate  criterion.  For  this  model  we  chose  to  use  D-Optimality. 


Figure  6:  Optimality  Criterion  selection 


Ensure  Run  Order  is  set  to  Randomize.  This  is  found  under  Output  Options.  Now  select  Make 
Table. 
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Figure  7:  Design  Matrix 


Step  5 

After  deleting  the  Y  column,  the  design  matrix  is  now  available  to  exported  to  Excel  or  saved 
as  a  txt  file  for  import  into  a  simulation  model. 


Figure  8:  Export  to  Excel 


Select  export  format. 
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